Ta có:
$\quad \sin^2\alpha + \cos^2\alpha = 1$
$\Leftrightarrow \cos^2\alpha = 1- \sin^2\alpha$
$\Leftrightarrow \cos^2\alpha = 1 - \dfrac{25}{169}$
$\Leftrightarrow \cos^2\alpha = \dfrac{144}{169}$
$\Rightarrow \cos\alpha = -\dfrac{12}{13}\quad \left(Do\ \dfrac{\pi}{2} < \alpha< \pi\right)$
$\Rightarrow \begin{cases}\tan\alpha =\dfrac{\dfrac{5}{13}}{-\dfrac{12}{13}}=- \dfrac{5}{12}\\\cot\alpha = -\dfrac{12}{5}\end{cases}$
Khi đó ta được:
$\quad B =\dfrac{\cos^2\alpha + \cot^2\alpha}{\tan\alpha - \cot\alpha}$
$\to B = \dfrac{\dfrac{144}{169} + \dfrac{144}{25}}{- \dfrac{5}{12} + \dfrac{12}{5}}$
$\to B = \dfrac{335\ 232}{100\ 555}$