Ta có \(P=(a^2+b^2-c^2)\left [ \frac{1}{(a-c)^2}+\frac{1}{(b-c)^2} \right ]-\frac{c^2}{a^2+b^2}+1\)
Do \(a^3+b^3=c^3\Leftrightarrow \left ( \frac{a}{c} \right )^2+\left ( \frac{b}{c} \right )^2=1 \Rightarrow 0< \frac{a}{c}< 1;0< \frac{b}{c}< 1\)
Do đó \(1=\left ( \frac{a}{c} \right )^2+\left ( \frac{b}{c} \right )^2< \left ( \frac{a}{c} \right )^2 +\left ( \frac{b}{c} \right )^2\Rightarrow a^2+b^2-c^2>0 \ (1)\)
Theo BĐT Cô Si: \(\frac{1}{(a-c)^2}+\frac{1}{(b-c)^2}\geq 2\sqrt{\frac{1}{(a-c)^2}.\frac{1}{(b-c)^2}} =\frac{2}{(c-a)(c-b)}>0 \ (2)\)
Từ (1) và (2) suy ra
\(P\geq \frac{2(a^2+b^2-c^2)}{(c-a)(c-b)}-\frac{c^2}{a^2+b^2}+1 \ (1)\)
Đặt \(x=\frac{a}{c};y=\frac{b}{c};t=x+y\). Ta có \(x^3+y^3=1\)
Dễ thấy \(x^3+y^3=(x+y)\left [ \frac{1}{4}(x+y)^2+\frac{3}{4}(x-y)^2 \right ]\geq \frac{1}{4 }(x+y)^3\Rightarrow x+y\leq \sqrt[3]{4}\)và \((x+y)^3>x^3+y^3=1\Rightarrow x+y>1\) nên \(t\in (1;\sqrt[3]{4}]\)

Ta có \(x^3+y^3=(x+y)^3-3xy(x+y)\Rightarrow 1=t^3-3txy\Rightarrow xy=\frac{t^3-1}{3t}\)
\(\Rightarrow x^2+y^2=(x+y)^2-2xy=\frac{t^3+2}{3t}\). Từ (3) suy ra:
\(P\geq \frac{2(x^2+y^2-1)}{(1-x)(1-y)}-\frac{1}{x^2+y^2}\)
\(=\frac{2(t+2)(t-1)^2}{(t-1)^3}-\frac{3t}{t^3+2}+1=\frac{2(t+2)}{t-1}-\frac{3t}{t^3+2}+1\)
Xét hàm số \(f(t)=\frac{2(t+2)}{t-1}-\frac{3t}{t^3+2}+1;t\in (1;\sqrt[3]{4}]\)
\(f'(t)=\frac{-6}{(t-1)^2}+\frac{6t^3}{(t^3+2)^2}=\frac{6\left [ (t-1)^2(t^3-1)-(t^3+2)^2 \right ]}{(t-1)^2(t^3+2)^2}<0\)
(Vì \(t\in (1;\sqrt[3]{4})\Rightarrow (t-1)^2(t^3-1)\leq 3(\sqrt[3]{4}-1)^2<2(t^3+2)^2>2)\)
Suy ra \(\underset{(1;\sqrt[3]{4}]}{min}f(t)=f(\sqrt[3]{4})=\frac{5\sqrt[3]{4}+6}{2(\sqrt[3]{4}-1)}\)
Từ đó ta có \(minP=\frac{5\sqrt[3]{4}+6}{2(\sqrt[3]{4}-1)}\) khi a = b = \(a=b=\frac{\sqrt[3]{4}}{2}c\)