I want to use mathematica to symbolically solve the minimization problem in simple linear regression:

$${\text{Find }}\text{arg}\min _{\alpha ,\,\beta }Q(\alpha ,\beta ),\qquad {\text{for }}Q(\alpha ,\beta ) =\sum _{i=1}^{n}(y_{i}-\alpha -\beta x_{i})^{2},$$

where $y_i$, $x_i$, and $n$ are symbolic but not specific numbers.The expected answer would be something like

$$\begin{align} \hat {\beta }&={\frac {\sum _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}{\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}}},\\ {\hat {\alpha }}&={\bar {y}}-{\hat {\beta }}\,{\bar {x}} \end{align}$$

Can someone show some code example for doing this? Thanks in advance!

  • $\begingroup$ Thanks Michael for your suggestions! $\endgroup$
    – CrossD
    Jul 9, 2016 at 21:49
  • $\begingroup$ See (107426). $\endgroup$
    – MarcoB
    Jul 9, 2016 at 21:54
  • $\begingroup$ What have you done so far? People are more likely to help if you show effort. $\endgroup$
    – Feyre
    Jul 9, 2016 at 21:56
  • 1
    $\begingroup$ You're welcome. Maybe someone else can help with your question. My experience has been that Mathematica's facility with algebraic/symbolic sums is limited. $\endgroup$
    – Michael E2
    Jul 9, 2016 at 21:56

1 Answer 1


The following is an adaptation of my answer to this question, which focused on the 3D linear least-squares problem.

(* Rules to get constants out of sums (or integrals etc) *)
outrules = {
   Sum[f_ + g_, it : {x_Symbol, __}] :> Sum[f, it] + Sum[g, it],
   Sum[c_ f_, it : {x_Symbol, __}] :> c Sum[f, it] /; FreeQ[c, x],
   Sum[c_, it : {x_Symbol, __}] :> c Sum[1, it] /; FreeQ[c, x]

(*Generate the sum of squares*)
Sum[Expand[(y[i] - a x[i] - b)^2], {i, 1, n}];

(*Calculate the derivatives*)
Grad[%, {a, b}];

(*Use the linearity property of sums*)
Distribute /@ %;

(*Pull out any constants from summations*)
% //. outrules;

(*Set the derivatives equal to zero to generate a system of equations*)
Simplify[Thread[% == 0]];

(*Solve for the a, b parameters*)
Solve[%, {a, b}] // FullSimplify

Mathematica graphics

These sums are not yet expressed as a function of $(\bar{x},\bar{y})$, i.e. the average values of the $(x_i,y_i)$, respectively, as you have them in your question, but that should be a question of some algebraic transformations. Some such transformations are reported e.g. in this MathWorld document of least-squares fitting.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.