# Compute linear regression estimators with the moments method

I would like to solve this system

hyp1 = Sum[Subscript[y, i] - Subscript[\[Beta], 0] - Subscript[\[Beta], 1], {i,1, n}]
hyp2 = Sum[Subscript[x,i]*(Subscript[y, i] - Subscript[\[Beta], 0] - Subscript[\[Beta],1]), {i, 1, n}]
Solve[{hyp1 == 0, hyp2 == 0}, {Subscript[\[Beta], 0],Subscript[\[Beta], 1]}]


But

This system cannot be solved with the methods available to Solve.

What function should I use? Thanks

• I think you want to multiply $\beta_1$ by $x_i$ in both equations. Also so that you see what Mathematica can and cannot do, you might try fixing $n=10$ (or some other positive integer greater than 2) and see what comes out. Otherwise, $\beta_0+\beta_1$ is really just a single parameter.
– JimB
Commented Aug 25, 2016 at 16:15
• @Artes, do you know how to do it in Mathematica? Commented Aug 25, 2016 at 17:40
• @Jim Baldwin, I'll try restraining n, thanks for the constructive answer. Commented Aug 25, 2016 at 17:41

Mathematica seems to make this unreasonably difficult, and won't automatically move factors outside sums or distribute them over Plus. Here is a way to solve the equations for generic n.

First define our equations (I've added $x_i$ as spotted by @Jim Baldwin), simplifying as we go

dist = Distribute[#, Plus, Sum] &
rule = HoldPattern[Sum[u_?(FreeQ[#, i] &) v_, w__]] :> u Sum[v, w]
hyp1 = (Sum[Subscript[y, i] - b0 - b1 Subscript[x, i], {i, 1, n}] //
dist) //. rule
hyp2 = (Sum[
Expand[Subscript[x,
i]*(Subscript[y, i] - b0 - b1 Subscript[x, i])], {i, 1, n}] //
dist) //. rule


Then there is no trouble in solving the equations

Solve[{hyp1 == 0, hyp2 == 0}, {b0, b1}] // Simplify


I think you have to be pretty dedicated to go through this, but I have found it useful in the past and have derived results that are far more complicated.

Edit

To understand the effect of these various functions, consider one of our starting expressions

Sum[Subscript[x, i]*(Subscript[y, i] - b0 - b1 Subscript[x, i]), {i,
1, n}]


The problem is that the coefficients we wish to solve for (b0 and b1) are inside the summation, where Solve does not have access to them. Our first step is to expand the expression inside the summation. We could do this as

% /. Sum[u_, v_] :> Sum[Expand[u], v]


We now need to split this into a number of sums. This can be done using

Distribute[%, Plus, Sum]


We can now take the factors independent of i outside the summation

% /. Sum[u_?(FreeQ[#, i] &) v_, w__] :> u Sum[v, w]


This is now in a form where Solve has access to the variables of interest.

(Note have I avoided subscripted variables everywhere they are not strictly necessary, because I generally find them more trouble than they are worth).

• Your solution works nicely, but unfortunately I'm too much of a beginner to understand this code. Particularly, what is the use of HoldPattern here? Thanks Commented Aug 25, 2016 at 22:03
• You can find out by omitting it! If you do, you'll discover that it doesn't actually make any difference (in this case). HoldPattern means that it doesn't get evaluated before Mathematica tries to match it. In this case I was being over-cautious, but sometimes evaluating the summation will change its form. Commented Aug 25, 2016 at 22:21
• Ok so what's making a difference here is that you specify to the software that it has to distribute in the ordre "Plus" and then "Sum"? That's it? Thanks for your help btw Commented Aug 26, 2016 at 17:29
• @EBassal I've added more explanation - though you've probably worked it out for yourself by now. Commented Aug 26, 2016 at 17:51