0
$\begingroup$

I was experimenting polynomial gradient descent on Mathematica, but it got stuck on partial derivative calculation. Formula below:

Traditional Form:
$$ mse = \frac{\sum_{i=1}^{n}\left(y[[i]] - \left(a\times x[[i]]^2+b\times x[[i]]+c\right) \right)^2}{n} \\ grad = \{D[mse, a], D[mse, b], D[mse, c]\} $$

Input Form:

mse = Sum[y[[i]] - (a x[[i]]^2 + b x[[i]] + c), {i, 1, n}]^2 / n;
grad = {D[mse, a], D[mse, b], D[mse, c]}

polynomial gradient descent As the picture showed above, I'm able to get expected behavior if I write the partial derivatives by hand, Mathemathca seems can't get the partial derivatives for me for the MES loss gradient.

Is there a way to solve this issue ?

$\endgroup$
3
  • 8
    $\begingroup$ posting plain text Mathematica code (InputForm) would be better. $\endgroup$
    – Nasser
    Commented Dec 3, 2021 at 13:09
  • $\begingroup$ What code have you tried, and what difficulties have you encountered? $\endgroup$
    – bbgodfrey
    Commented Dec 3, 2021 at 15:18
  • $\begingroup$ I've updated the question, I don't know if it's too complex or I made some mistake $\endgroup$
    – Frei Zhang
    Commented Dec 3, 2021 at 15:32

1 Answer 1

1
$\begingroup$

So, I've managed to solve this issue, it's mainly due to my malformed input which cause Mathematica stuck. I leave it here for anyone who have the similar issue.

Below is the minimal reproducible test case(still tediously long). You'll found there is a minor difference(there is an empty superscript after a[[i]]) but not visually noticeable, and my InputForm rewrote in the question was also incorrect(mse = Sum[y[[i]] - (a x[[i]]^2 + b x[[i]] + c), {i, 1, n}]^2 / n;) /facepalm. diff

derivativeTest[{x_, y_}] := 
  Module[{ n, mseOrig, mseStandard, mseInput, gradInput, 
    gradStandard, gradSelf},
            (*MSE loss*)
            n = Length[x];
   mseOrig = 1/n \!\(
\*UnderoverscriptBox[\(\[Sum]\), \(i = 1\), \(n\)]
\*SuperscriptBox[\((y[\([\)\(i\)\(]\)]\  - \ \((a\ \ 
\*SuperscriptBox[\(x[\([\)\(i\)\(]\)]\), \(\ \)] x[\([i]\)] + \ 
           b\ x[\([\)\(i\)\(]\)]\  + \ c)\)\ )\), \(2\)]\)  ;
           mseStandard = 1/n \!\(
\*UnderoverscriptBox[\(\[Sum]\), \(i = 1\), \(n\)]
\*SuperscriptBox[\((y[\([\)\(i\)\(]\)]\  - \ \((a\ \ x[\([i]\)] \
x[\([i]\)] + \ b\ x[\([\)\(i\)\(]\)]\  + \ c)\)\ )\), \(2\)]\)  ;
   mseInput = 
    1/n  * Sum[(y[[i]] - (a x[[i]]^2 + b x[[i]] + c))^2, {i, 1, n}];
   
   gradInput = {D[mseInput, a], D[mseInput, b], D[mseInput, c]};
   gradStandard = {D[mseStandard, a], D[mseStandard, b], 
     D[mseStandard, c]};
   
         gradSelf = {
               -2/n \!\(
\*UnderoverscriptBox[\(\[Sum]\), \(i = 1\), \(n\)]\(
\*SuperscriptBox[\(x[\([\)\(i\)\(]\)]\), \(2\)] \((y[\([i]\)]\  - \ \
\((a\ 
\*SuperscriptBox[\(x[\([\)\(i\)\(]\)]\), \(2\)] + b\ x[\([i]\)]\  + \ 
            c)\))\)\)\),
               -2/n \!\(
\*UnderoverscriptBox[\(\[Sum]\), \(i = 
         1\), \(n\)]\(x[\([i]\)] \((y[\([i]\)]\  - \ \((a\ 
\*SuperscriptBox[\(x[\([\)\(i\)\(]\)]\), \(2\)] + b\ x[\([i]\)]\  + \ 
            c)\))\)\)\),
               -2/n \!\(
\*UnderoverscriptBox[\(\[Sum]\), \(i = 
         1\), \(n\)]\((y[\([i]\)]\  - \ \((a\ 
\*SuperscriptBox[\(x[\([\)\(i\)\(]\)]\), \(2\)] + b\ x[\([i]\)]\  + \ 
           c)\))\)\)
               };
    
    Grid[{{"MSE Original", "MSE StandardForm",  
       "MSE InputForm"}, {mseOrig // Simplify, 
       mseStandard // Simplify , mseInput // Simplify}}, Frame -> All]
    Grid[{{"Gradient by StandardForm", "Gradient by InputForm", 
       "Gradient by hand"}, {  gradStandard // Simplify, 
       gradInput // Simplify , gradSelf // Simplify }}, Frame -> All]
            ];
$\endgroup$
1
  • $\begingroup$ Still another reason never to use Subscript and the like when trying to do computations. $\endgroup$
    – bbgodfrey
    Commented Dec 4, 2021 at 15:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.