# How would I solve this math problem on Mathematica?

$$(3t^2 + 5t + a ) (4t^2 + bt - 2) = 12t^4 +26t^3 - 8t^2 - 16t + 6$$

What is a + b?

$$6=-2a$$
$$a=-3$$
$$-16t=(5t\cdot-2)+a\cdot bt\Longrightarrow-16t=-10t+(-3)bt\Longrightarrow b=2$$
$$a+b=-1$$

SolveAlways[] makes quick work of your problem:

SolveAlways[(3 t^2 + 5 t + a) (4 t^2 + b t - 2) == 12 t^4 + 26 t^3 - 8 t^2 - 16 t + 6, t]
{{a -> -3, b -> 2}}


One way could be

ClearAll[t,a,b}
z1 = CoefficientList[(3*t^2 + 5*t + a)*(4*t^2 + b*t - 2), t];
z2 = CoefficientList[12*t^4 + 26*t^3 - 8*t^2 - 16*t + 6, t];

Reduce[ForAll[t, (3*t^2 + 5*t + a)*(4*t^2 + b*t - 2) == 12*t^4 + 26*t^3 - 8*t^2 - 16*t + 6], a]