Methods for NonlinearModelFit

I am using NonlinearModelFit for a thesis project.

I get quite different results if I change the Method to LevenbergMarquardt or QuasiNewton or ConjugateGradient and so on. My nonlinear model includes polynomials up to second order and a sigmoid function such as ArcTan or Tanh. I would like to know, which conditions Mathematica uses to choose the best algorithm if I set the Method as Automatic. It works sometimes, but most of the time it doesn't and I do not have the knowledge to say which method should be the best in my case. Just by trying out, I found Gradient and sometimes ConjugateGradient to work the best for my purpose.

I also cannot find anywhere on the Internet which exact implementation of these methods is used inside Mathematica and how does it change the results. Here maybe an interesting link: Optimization Algorithms

Thanks!

• If necessary I can post some data and some output from Mathematica. Mar 22, 2013 at 10:15
• Have you read this reference.wolfram.com/mathematica/tutorial/…? Mar 22, 2013 at 10:27
• No, I hadn't. Thanks for the link! Probably this will explain some things, but still I think there is nowhere an explanation in the context of Nonlinear Model Fit. Mar 22, 2013 at 14:13
• To the contrary -- this tutorial (note there is also a pdf version you can download and read at your convenience) describes all of the optimization technologies -- of which NonlinearModeFit is just one. wolfram.com/learningcenter/tutorialcollection/… Apr 27, 2013 at 6:33
• NonlinearModelFit seems to be implemented as a wrapper around FindFit, but the latter is kernel code and not accessible for inspection. I would suggest contacting WRI support to ask them which method Automatic translates to and under what conditions. May 2, 2013 at 17:15

Unfortunately the Method option is not documented in detail on the NonlinearModelFit documentation page. To summarize what we know so far (comments, documentation, etc.):

NonlinearModelFit can either use numerical local optimization, or numerical global optimization.

• Local optimization is the same as used by FindMinimum and related functions. The possible method options are documented in detail here. NonlinearModelFit will take these options directly. Example:

NonlinearModelFit[..., Method -> {"Newton", "StepControl" -> "TrustRegion"}]

• Global optimization is the same as used by NMinimize and related functions. The available method options are detailed here. The syntax to use is

NonlinearModelFit[..., Method -> {NMinimize, Method -> ...}]


Further documentation on the constrained optimization is here.