Đặt $\sqrt{4x+5}=t(t\ge 0)$
$\Rightarrow x=\dfrac{t^2-5}{4}$
Phương trình trở thành:
$\begin{array}{l} 2{\left( {\dfrac{{{t^2} - 5}}{4}} \right)^2} - 6.\left( {\dfrac{{{t^2} - 5}}{4}} \right) - 1 = t\\ \Leftrightarrow \dfrac{{{{\left( {{t^2} - 5} \right)}^2}}}{8} - \dfrac{{3\left( {{t^2} - 5} \right)}}{2} - 1 = t\\ \Leftrightarrow {\left( {{t^2} - 5} \right)^2} - 12\left( {{t^2} - 5} \right) - 8 = 8t\\ \Leftrightarrow \left( {{t^4} - 10{t^2} + 25} \right) - 12{t^2} + 60 - 8t - 8 = 0\\ \Leftrightarrow {t^4} - 22{t^2} - 8t + 77 = 0\\ \Leftrightarrow \left( {{t^2} - 2t - 11} \right)\left( {{t^2} + 2t - 7} \right) = 0\\ \Leftrightarrow \left[ \begin{array}{l} t = 1 + 2\sqrt 3 \\ t = 1 - 2\sqrt 3 \\ t = - 1 + 2\sqrt 2 \\ t = - 1 - 2\sqrt 2 \end{array} \right.\\ \Rightarrow \left[ \begin{array}{l} t = 1 + 2\sqrt 3 \\ t = - 1 + 2\sqrt 2 \end{array} \right.\left( {t \ge 0} \right)\\ \Leftrightarrow \left[ \begin{array}{l} 4x + 5 = 13 + 4\sqrt 3 \\ 4x + 5 = 9 - 4\sqrt 2 \end{array} \right.\\ \Leftrightarrow \left[ \begin{array}{l} x = 2 - \sqrt 3 \\ x = 1 - \sqrt 2 \end{array} \right.\\ \Rightarrow S = \left\{ {2 - \sqrt 3 ;1 - \sqrt 2 } \right\} \end{array}$