I am trying to solve the simple linear regression equation analytically in Mathematica (not using builtin functions). I managed to calculate the gradient as follows f is loss function using RSS and weights w0 and w1:

f[w0_, w1_] := Sum[(Subscript[y, i] - (w0 + w1*Subscript[x, i]))^2, {i, 0, n}]
Grad[f[w0, w1], {w0, w1}]

The solution is correctly output as

{Sum[-2*(-w0 - w1*Subscript[x, i] + Subscript[y, i]), {i, 0, n}], 
  Sum[-2*Subscript[x, i]*(-w0 - w1*Subscript[x, i] + Subscript[y, i]), 
   {i, 0, n}]}

Now how to solve the case where I need to set the Gradient of vectors equals 0 to get the analytic solutions for w0 and w1. The analytic solution to the equation is (N=n)

w1 = (Sum[Subscript[x, i]*Subscript[y, i], {i, 0, n}] - 
    Sum[Subscript[x, i]*Sum[Subscript[y, i]/N, {i, 0, n}], {i, 0, n}])/
   (Sum[Subscript[x, i]^2, {i, 0, n}] - 
    Sum[Subscript[x, i]*Sum[Subscript[x, i]/N, {i, 0, n}], {i, 0, n}])

and for w0 is

w0 = (Sum[Subscript[y, i], {i, 0, n}] - w1*Sum[Subscript[x, i], {i, 0, n}])/N

I tried Solve to solve this system of equations but without any success. Can you kindly help me to do this in Mathematica or is it not possible

  • $\begingroup$ which variable you want to solve it for? $\endgroup$ – Sumit Jun 17 '16 at 11:22
  • $\begingroup$ w0 and w1. I want to get the weights analytically. the answers fro w0 and w1 are the actual solutions $\endgroup$ – arvind Jun 17 '16 at 11:25
  • $\begingroup$ Please take a look at this: meta.mathematica.stackexchange.com/questions/1584/… and repost your code in text-only form. As it is, your code is entirely unreadable. $\endgroup$ – MarcoB Jun 17 '16 at 14:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.