# Difference between Fitting Algorithms

I am new to Mathematica, and I don´t quite understand, what the difference between the fitting algorithms Fit, FindFit, LinearModelFit, NonLinearModelFit are. Why is the syntax so different ({1,x,x^2} vs. a + b x + c x^2)? Which should I prefer for which use-case?
Why can I use a non linear Function like a + b x + c x^2 in an LinearModelFit? I am very confused, I hope you can help me.

• Well the difference between Linear and NonLinear should be clear given the name? :-) – chris Apr 27 '15 at 16:55
• Fit and FindFit differ in the way they handle the result. One is meant to cut the chase and give the result, the other provide with the whole statistical analysis. – chris Apr 27 '15 at 16:55
• some of the differences are historical. – chris Apr 27 '15 at 16:56
• FindFit \ Properties and Relations section explains a lot. But not everything. And I think question is on topic, at the beginning I was always confused - "great that I can do this in 10 different ways, but which one is appriopriate?" – Kuba Apr 27 '15 at 17:19
• Strongly related – bobthechemist Apr 28 '15 at 1:49

Luca, I think that you may be misunderstanding the meaning of "linear" in the names of those functions, and in general in reference to linear models.

Just to make sure that we agree on the nomenclature here, in a fitting model you have parameters, and predictor variables. For instance, in a simple quadratic model such as y = a x^2 + b x + c we typically understand a, b, c as the parameters, and x as the predictor variable.

Linear models are those models in which the predicted value (e.g. y in our example) depends linearly on the parameters of the model. It doesn't matter whether the relationship between y and x is linear. For instance, the model above is linear. This following model is also linear: y = a Sin[x^2]. Model fitting is a search for the parameters, so linear parameter relationships lead to models that are easier to deal with computationally, and in some cases to closed-form solutions to the regression problem.

Non-linear models are those in which the predicted value depends on the parameters in a non-linear fashion. For instance: y = Sin[a] x is non linear because the predicted value depends non-linearly on the value of the parameter a (even though the predicted value depends linearly on x!).

In addition to the interesting reasons mentioned by the commenters to your question, which I think shed some light on the issue as well, the existence of multiple fitting functions also reflects this fundamental divide.

On the basis of my (limited) experience, here is how I view the most common fitting functions:

• Fit is the oldest of the bunch; it carries out linear model fitting by linearly combining a list of base functions you explicitly indicate. I don't tend to use it much, since the majority of models I fit to are non-linear.
• FindFit is newer; it is capable of both linear and non-linear fitting. You feed it the explicit form of the fitting model directly. I find this very intuitive to use.
• NonlinearModelFit accomplishes the same results as FindFit and it shares the same syntax. In addition to the fitted model, it also gives you access to a wealth of fitting descriptors: goodness of fit estimates, prediction intervals, residuals, just to name a few. You could think of it as a FindFit "on steroids". It would be really worth your time to take a look at the many options available for this function ("Details and Options" section of its documentation).
• FindFit is also faster than NonlinearModelFit: the former is implemented in fast numerical code within the kernel; the latter is written entirely in top-level code, i.e. using Mathematica expressions, and it internally calls FindFit to do the heavy lifting anyway (@OleksandrR. talked about this point more authoritatively in his answer regarding compiled versions of the fitting routines).
• Oh ok, my fault. And can I use Fit and FindFit also for non linear fitting? – Luca Thiede Apr 27 '15 at 18:02
• @LucaThiede Good point. Let me add something to that respect to the answer. – MarcoB Apr 27 '15 at 18:25
• Ok, and I assume that LinearModelFit is the same as NonLinearModelFit, but limited to the Linear Models, whereas NonLinearModelFit is for everything, is that correct? Has LinearModelFit any advantage over the NonLinearModelFit? – Luca Thiede Apr 27 '15 at 18:40
• @LucaThiede You are correct on the Linear vs Nonlinear functions. Linear models can typically be solved deterministically (i.e. there is a closed-form solution to find the best fit parameters), whereas non-linear models require an iterative process, so if your model is linear the functions specifically geared to the linear models are going to be faster and more reliable. Of course, if your model is not linear, then you are stuck with the iterative processes. – MarcoB Apr 27 '15 at 18:47