Lời giải:
Với điều kiện đã cho thì hiển nhiên mẫu dương.
Áp dụng BĐT Cauchy-Schwarz ta có:
\(M=\frac{a^2}{2a\sqrt{b}-3a}+\frac{b^2}{2b\sqrt{c}-3b}+\frac{c^2}{2c\sqrt{a}-3c}\)\(\geq \frac{(a+b+c)^2}{2(a\sqrt{b}+b\sqrt{c}+c\sqrt{a})-3(a+b+c)}\)
Áp dụng BĐT Bunhiacopxky kết hợp BĐT AM-GM:
\((a\sqrt{b}+b\sqrt{c}+c\sqrt{a})^2\leq (a+b+c)(ab+bc+ac)\)
\(\leq (a+b+c).\frac{(a+b+c)^2}{3}=\frac{(a+b+c)^3}{3}\)
\(\Rightarrow a\sqrt{b}+b\sqrt{c}+c\sqrt{a}\leq \sqrt{\frac{(a+b+c)^3}{3}}\)
\(\Rightarrow M\geq \frac{(a+b+c)^2}{2\sqrt{\frac{(a+b+c)^3}{3}}-3(a+b+c)}\)
Đặt \(\sqrt{\frac{a+b+c}{3}}=t(t>\frac{3}{2})\)\(\Rightarrow a+b+c=3t^2\)
Ta có:
\(P\geq\frac{9t^4}{6t^3-9t^2}=\frac{3t^2}{2t-3}\)
\(\Leftrightarrow P\geq \frac{\frac{3}{4}(2t-3)(2t+3)}{2t-3}+\frac{27}{4(2t-3)}\)
\(\Leftrightarrow P\geq \frac{3}{4}(2t+3)+\frac{27}{4(2t-3)}=\frac{3}{4}(2t-3)+\frac{27}{4(2t-3)}+\frac{9}{2}\)
Áp dụng BĐT AM-GM:
\(\frac{3}{4}(2t-3)+\frac{27}{4(2t-3)}\geq 2\sqrt{\frac{3}{4}.\frac{27}{4}}=\frac{9}{2}\)
\(\Rightarrow P\geq \frac{9}{2}+\frac{9}{2}=9\)
Vậy \(P_{\min}=9\)
