# How to symbolically solve the minimization problem in simple linear regression

I want to use mathematica to symbolically solve the minimization problem in simple linear regression:

$${\text{Find }}\text{arg}\min _{\alpha ,\,\beta }Q(\alpha ,\beta ),\qquad {\text{for }}Q(\alpha ,\beta ) =\sum _{i=1}^{n}(y_{i}-\alpha -\beta x_{i})^{2},$$

where $y_i$, $x_i$, and $n$ are symbolic but not specific numbers.The expected answer would be something like

\begin{align} \hat {\beta }&={\frac {\sum _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}{\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}}},\\ {\hat {\alpha }}&={\bar {y}}-{\hat {\beta }}\,{\bar {x}} \end{align}

Can someone show some code example for doing this? Thanks in advance!

• Thanks Michael for your suggestions! Jul 9 '16 at 21:49
• See (107426). Jul 9 '16 at 21:54
• What have you done so far? People are more likely to help if you show effort. Jul 9 '16 at 21:56
• You're welcome. Maybe someone else can help with your question. My experience has been that Mathematica's facility with algebraic/symbolic sums is limited. Jul 9 '16 at 21:56

The following is an adaptation of my answer to this question, which focused on the 3D linear least-squares problem.

(* Rules to get constants out of sums (or integrals etc) *)
outrules = {
Sum[f_ + g_, it : {x_Symbol, __}] :> Sum[f, it] + Sum[g, it],
Sum[c_ f_, it : {x_Symbol, __}] :> c Sum[f, it] /; FreeQ[c, x],
Sum[c_, it : {x_Symbol, __}] :> c Sum[1, it] /; FreeQ[c, x]
};

(*Generate the sum of squares*)
Sum[Expand[(y[i] - a x[i] - b)^2], {i, 1, n}];

(*Calculate the derivatives*)

(*Use the linearity property of sums*)
Distribute /@ %;

(*Pull out any constants from summations*)
% //. outrules;

(*Set the derivatives equal to zero to generate a system of equations*)

These sums are not yet expressed as a function of $(\bar{x},\bar{y})$, i.e. the average values of the $(x_i,y_i)$, respectively, as you have them in your question, but that should be a question of some algebraic transformations. Some such transformations are reported e.g. in this MathWorld document of least-squares fitting.