# Methods for NonlinearModelFit

I am using NonlinearModelFit for a thesis project.

I get quite different results if I change the Method to LevenbergMarquardt or QuasiNewton or ConjugateGradient and so on. My nonlinear model includes polynomials up to second order and a sigmoid function such as ArcTan or Tanh. I would like to know, which conditions Mathematica uses to choose the best algorithm if I set the Method as Automatic. It works sometimes, but most of the time it doesn't and I do not have the knowledge to say which method should be the best in my case. Just by trying out, I found Gradient and sometimes ConjugateGradient to work the best for my purpose.

I also cannot find anywhere on the Internet which exact implementation of these methods is used inside Mathematica and how does it change the results. Here maybe an interesting link: Optimization Algorithms

Thanks!

-
If necessary I can post some data and some output from Mathematica. –  Santi Mar 22 '13 at 10:15
Have you read this reference.wolfram.com/mathematica/tutorial/…? –  belisarius Mar 22 '13 at 10:27
No, I hadn't. Thanks for the link! Probably this will explain some things, but still I think there is nowhere an explanation in the context of Nonlinear Model Fit. –  Santi Mar 22 '13 at 14:13
To the contrary -- this tutorial (note there is also a pdf version you can download and read at your convenience) describes all of the optimization technologies -- of which NonlinearModeFit is just one. wolfram.com/learningcenter/tutorialcollection/… –  bill s Apr 27 '13 at 6:33
NonlinearModelFit seems to be implemented as a wrapper around FindFit, but the latter is kernel code and not accessible for inspection. I would suggest contacting WRI support to ask them which method Automatic translates to and under what conditions. –  Oleksandr R. May 2 '13 at 17:15

Unfortunately the Method option is not documented in detail on the NonlinearModelFit documentation page. To summarize what we know so far (comments, documentation, etc.):

NonlinearModelFit can either use numerical local optimization, or numerical global optimization.

• Local optimization is the same as used by FindMinimum and related functions. The possible method options are documented in detail here. NonlinearModelFit will take these options directly. Example:

NonlinearModelFit[..., Method -> {"Newton", "StepControl" -> "TrustRegion"}]

• Global optimization is the same as used by NMinimize and related functions. The available method options are detailed here. The syntax to use is

NonlinearModelFit[..., Method -> {NMinimize, Method -> ...}]


Further documentation on the constrained optimization is here.

-