a)
`x^2-3x-40=0`
`<=>x^2+5x-8x-40=0`
`<=>x(x+5)-8(x+5)=0`
`<=>(x-8)(x+5)=0`
`<=>`\(\left[ \begin{array}{l}x-8=0\\x+5=0\end{array} \right.\)
`<=>`\(\left[ \begin{array}{l}x=8\\x=-5\end{array} \right.\)
Vậy: `S={8;-5}`
b)
`5x(x+1)-39(x+1)-3(x+1)(x-1)=2x^2`
`<=>(5x-39-3x+3)(x+1)=2x^2`
`<=>(2x-36)(x+1)=2x^2`
`<=>2x^2+2x-36x-36=2x^2`
`<=>2x^2-2x^2-34x-36=0`
`<=>-34x-36=0`
`<=>x=-18/17`
Vậy: `S={-18/17}`