Does FindFit use symbolic differentiation when optimizing by least squares a non-linear function?


FindFit does perform symbolic differentiation by default. It has the option to use finite differences instead via the option Gradient -> "FiniteDifference".

There is worked example in the documentation here.


Yes it does. You can see this by using a function that prints the argument passed to the function evaluated in the FindFit procedure. For example you could use this little setup:

data = Table[{x, 2.5*Exp[x]}, {x, 0, 1, 0.1}];
f[a_, x_] := Block[{}, Print[x]; a*Exp[x]]
FindFit[data, f[a, x], {a}, x]

What you will notice, when you execute the code, is that in addition to the correct result a = 2.5 the Print routine yields the output x. This means that the function f has been evaluated symbolically by Mathematica, otherwise you would have gotten a list of numerical values, which would correspond to points where f would have been evaluated numerically.

  • 1
    $\begingroup$ Since FindFit is not HoldAll, before FindFit is executed, f[a, x] is evaluated and the result is passed to FindFit. I'm afraid this example does not show what FindFit does with the expression it gets. -- BTW, I think you're probably right about FindFit differentiating, but I don't know how to prove it as yet. $\endgroup$ – Michael E2 Feb 3 '15 at 21:58
  • $\begingroup$ Ok, I did not realise that. Thanks for the info. I will leave this answer up for now, because I think your comment is interesting, but I am glad to remove it. Sorry for the misinformation. $\endgroup$ – Wizard Feb 3 '15 at 22:08
  • $\begingroup$ I'm not sure if derivatives are used, but you get slight differences using a symbolic function and a numeric black box. For example, f[a_, x_] := a*Exp[x]; g[a_?NumericQ, x_?NumericQ] := a*Exp[x]; yield values of a that differ by 10^15. The symbolic one is about ten times faster on your example. It suggests derivatives, imo, but perhaps it's not conclusive. $\endgroup$ – Michael E2 Feb 3 '15 at 22:21
  • $\begingroup$ Ok, now this example convinces me that Mathematica does not use symbolic differentiation: data = Table[{x, Pi*x}, {x, 0, 1, 1}]; FindFit[data, a*x, {a}, x] yields {a -> 3.14159} and not {a -> Pi}, which should be the case if Mathematica would differentiate symbolically. $\endgroup$ – Wizard Feb 6 '15 at 22:14
  • 1
    $\begingroup$ Pi == a /. FindFit[data, a*x, {a}, x] evaluates to True, though, and working precision is MachinePrecision. FindFit cannot (or will not) return exact, infinite precision answers. The result is the same as Pi to 150 digits if I use WorkingPrecision -> 150. $\endgroup$ – Michael E2 Feb 6 '15 at 22:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.