Đáp án:
$\begin{array}{l}
a)m = 6\\
\Leftrightarrow {x^2} + 5x + 8 = 0\\
\Leftrightarrow {x^2} + 2.x.\dfrac{5}{2} + \dfrac{{25}}{4} + \dfrac{7}{4} = 0\\
\Leftrightarrow {\left( {x + \dfrac{5}{2}} \right)^2} + \dfrac{7}{4} = 0\left( {vn} \right)
\end{array}$
Vậy pt vô nghiệm khi m=6
$\begin{array}{l}
b)\Delta \ge 0\\
\Leftrightarrow {5^2} - 4\left( {m + 2} \right) \ge 0\\
\Leftrightarrow m + 2 \le \dfrac{{25}}{4}\\
\Leftrightarrow m \le \dfrac{{17}}{4}\\
Theo\,Viet:\left\{ \begin{array}{l}
{x_1} + {x_2} = - 5\\
{x_1}{x_2} = m + 2
\end{array} \right.\\
A = {\left( {{x_1} - {x_2}} \right)^2} + 8{x_1}{x_2}\\
= {\left( {{x_1} + {x_2}} \right)^2} - 4{x_1}{x_2} + 8{x_1}{x_2}\\
= {\left( { - 5} \right)^2} + 4.\left( {m + 2} \right)\\
= 4m + 32\\
Do:m \le \dfrac{{17}}{4}\, \Leftrightarrow 4m + 32 \le 49\\
\Leftrightarrow A \le 49\\
\Leftrightarrow GTLN:A = 49\,khi:m = \dfrac{{17}}{4}
\end{array}$