Đặt \(\sqrt x = t \Rightarrow x = {t^2} \Rightarrow dx = 2tdt\)
\( \Rightarrow I = \int {t.{{\ln }^2}{t^2}.2tdt} = 2\int {{t^2}.{{\left( {\ln {t^2}} \right)}^2}dt} \)
\( = 2\int {{t^2}.{{\left( {2\ln t} \right)}^2}dt} = 8\int {{t^2}{{\ln }^2}tdt} \)
Đặt \(\left\{ \begin{array}{l}{\ln ^2}t = u\\{t^2}dt = dv\end{array} \right. \Rightarrow \left\{ \begin{array}{l}du = 2\ln t.\dfrac{1}{t}dt\\v = \dfrac{{{t^3}}}{3}\end{array} \right.\)
\( \Rightarrow \int {{t^2}{{\ln }^2}tdt} = \dfrac{{{t^3}}}{3}.{\ln ^2}t - \dfrac{2}{3}\int {{t^2}\ln tdt} \)
Đặt \(\left\{ \begin{array}{l}\ln t = u\\{t^2}dt = dv\end{array} \right. \Rightarrow \left\{ \begin{array}{l}du = \dfrac{1}{t}dt\\v = \dfrac{{{t^3}}}{3}\end{array} \right.\)
\( \Rightarrow \int {{t^2}\ln tdt} = \dfrac{{{t^3}\ln t}}{3} - \dfrac{1}{3}\int {{t^2}dt} = \dfrac{{{t^3}\ln t}}{3} - \dfrac{1}{9}{t^3} + C\)
\( \Rightarrow I = 8\left[ {\dfrac{{{t^3}{{\ln }^2}t}}{3} - \dfrac{2}{3}.\left( {\dfrac{{{t^3}\ln t}}{3} - \dfrac{1}{9}{t^3}} \right)} \right] + C\) \( = \dfrac{8}{3}{t^3}{\ln ^2}t - \dfrac{{16}}{3}\left( {\dfrac{{{t^3}\ln t}}{3} - \dfrac{{{t^3}}}{9}} \right) + C\) \( = \dfrac{8}{3}{t^3}{\ln ^2}t - \dfrac{{16}}{9}{t^3}\ln t + \dfrac{{16}}{{27}}{t^3} + C\) \( = \dfrac{8}{3}.\sqrt {{x^3}} .{\ln ^2}\sqrt x - \dfrac{{16}}{9}.\sqrt {{x^3}} .\ln \sqrt x + \dfrac{{16}}{{27}}.\sqrt {{x^3}} + C\)
\( = \dfrac{8}{{27}}x\sqrt x \left( {\dfrac{9}{4}{{\ln }^2}x - 3\ln x + 2} \right) + C\)
