`~rai~`
\(1.\cos^2x-\sqrt{3}\sin2x=1+\sin^2x\\\Leftrightarrow \sin^2x+2\sqrt{3}\sin x\cos x-\cos^2x=-1\quad(1)\\+)\text{Thay }\cos x=0(\sin x=\pm 1)\text{vào (1) được:}\\(1)\Leftrightarrow 1=-1.\text{(vô lí)}\\+)\text{Xét }\cos x\ne 0,\text{chia cả 2 vế phương trình (1) cho }\cos^2x\text{ được:}\\(1)\Leftrightarrow \tan^2x+2\sqrt{3}\tan x-1=-(\tan^2x+1)\\\Leftrightarrow 2\tan^2x+2\sqrt{3}\tan x=0\\\Leftrightarrow \left[\begin{array}{I}\tan x=0\\\tan x=\sqrt{3}\end{array}\right.\\\Leftrightarrow \left[\begin{array}{I}x=k\pi\\x=\dfrac{\pi}{3}+k\pi.\end{array}\right.\quad(k\in\mathbb{Z})\\\text{Vậy S=}\left\{k\pi;\dfrac{\pi}{3}+k\pi\Big|k\in\mathbb{Z}\right\}.\\2.\sin^2x-(\sqrt{3}+1)\sin x\cos x+\sqrt{3}\cos^2x=0\quad(1)\\+)\text{Thay }\cos x=0(\sin x=\pm 1)\text{ vào (1) được:}\\(1)\Leftrightarrow 1=0.\text{(vô lí)}\\+)\text{Xét }\cos x\ne 0,\text{chia cả 2 vế phương trình cho }\cos^2x\text{ được:}\\(1)\Leftrightarrow \tan^2x-(\sqrt{3}+1)\tan x+\sqrt{3}=0\\\Leftrightarrow \left[\begin{array}{I}\tan x=1\\\tan x=\sqrt{3}\end{array}\right.\\\Leftrightarrow \left[\begin{array}{I}x=\dfrac{\pi}{4}+k\pi\\x=\dfrac{\pi}{3}+k\pi.\end{array}\right.\quad(k\in\mathbb{Z})\\\text{Vậy S=}\left\{\dfrac{\pi}{3}+k\pi;\dfrac{\pi}{4}+k\pi\Big|k\in\mathbb{Z}\right\}.\)