Đáp án:

 Vậy phương trình có các họ nghiệm là:

$a)x = \dfrac{\pi }{4} + k\dfrac{\pi }{2};x = \dfrac{\pi }{{12}} + k\dfrac{\pi }{2}\left( {k \in Z} \right)$

$b)x = \dfrac{\pi }{4} + k\pi ;x = \arctan \left( { - 5} \right) + k\pi \left( {k \in Z} \right)$

Giải thích các bước giải:

$\begin{array}{l}
a)2{\cos ^2}2x - 3\sqrt 3 \sin 4x - 4{\sin ^2}2x =  - 4\\
 \Leftrightarrow 2{\cos ^2}2x - 6\sqrt 3 \sin 2x\cos 2x - 4{\sin ^2}2x =  - 4{\cos ^2}2x - 4{\sin ^2}2x\\
 \Leftrightarrow 6{\cos ^2}2x - 6\sqrt 3 \sin 2x\cos 2x = 0\\
 \Leftrightarrow 6\cos 2x\left( {\cos 2x - \sqrt 3 \sin 2x} \right) = 0\\
 \Leftrightarrow \left[ \begin{array}{l}
\cos 2x = 0\\
\cos 2x - \sqrt 3 \sin 2x = 0
\end{array} \right.\\
 \Leftrightarrow \left[ \begin{array}{l}
2x = \dfrac{\pi }{2} + k\pi \\
\dfrac{1}{2}\cos 2x - \dfrac{{\sqrt 3 }}{2}\sin 2x = 0
\end{array} \right.\left( {k \in Z} \right)\\
 \Leftrightarrow \left[ \begin{array}{l}
2x = \dfrac{\pi }{2} + k\pi \\
\cos \left( {2x + \dfrac{\pi }{3}} \right) = 0
\end{array} \right.\left( {k \in Z} \right)\\
 \Leftrightarrow \left[ \begin{array}{l}
x = \dfrac{\pi }{4} + k\dfrac{\pi }{2}\\
2x + \dfrac{\pi }{3} = \dfrac{\pi }{2} + k\pi 
\end{array} \right.\left( {k \in Z} \right)\\
 \Leftrightarrow \left[ \begin{array}{l}
x = \dfrac{\pi }{4} + k\dfrac{\pi }{2}\\
x = \dfrac{\pi }{{12}} + k\dfrac{\pi }{2}
\end{array} \right.\left( {k \in Z} \right)
\end{array}$

$\begin{array}{l}
b){\sin ^2}x + \sin 2x - 2{\cos ^2}x = \dfrac{1}{2}\\
 \Leftrightarrow 2{\sin ^2}x + 2\sin 2x - 4{\cos ^2}x = 1\\
 \Leftrightarrow 2{\sin ^2}x + 4\sin x\cos x - 4{\cos ^2}x = {\sin ^2}x + {\cos ^2}x\\
 \Leftrightarrow {\sin ^2}x + 4\sin x\cos x - 5{\cos ^2}x = 0\\
 \Leftrightarrow \left( {\sin x - \cos x} \right)\left( {\sin x + 5\cos x} \right) = 0\\
 \Leftrightarrow \left[ \begin{array}{l}
\sin x - \cos x = 0\\
\sin x + 5\cos x = 0
\end{array} \right.\\
 \Leftrightarrow \left[ \begin{array}{l}
\sin x = \cos x\\
\sin x =  - 5\cos x
\end{array} \right.\\
 \Leftrightarrow \left[ \begin{array}{l}
\tan x = 1\\
\tan x =  - 5
\end{array} \right.\\
 \Leftrightarrow \left[ \begin{array}{l}
x = \dfrac{\pi }{4} + k\pi \\
x = \arctan \left( { - 5} \right) + k\pi 
\end{array} \right.\left( {k \in Z} \right)
\end{array}$