# Using NonlinearModelFit to fit data with errors

Assume we have some data of measurements

SeedRandom[9];
data = Table[{i,2*(1 + 5*RandomReal[])*Exp[-0.01*(1 + RandomReal[])*i]}, {i, {0, 100, 200, 400, 700}}];


and measurement errors associated to it

errors = 10^# & /@ Range[0, -4, -1];


Ignoring the weights in a first step, the fitting works very well:

nlm = NonlinearModelFit[data, a*Exp[-b*x], {a, b}, x, Method -> "NMinimize"];
Show[ListPlot[data, PlotStyle -> PointSize -> Medium],
Plot[nlm[x], {x, 0, 700}, PlotRange -> Full], PlotRange -> All]


Now, according to this how-to, we can include the measurement errors to the fitting procedure:

nlm = NonlinearModelFit[data, a*Exp[-b*x], {a, b}, x, Weights -> 1/errors^2, VarianceEstimatorFunction -> (1 &), Method -> "NMinimize"];
Show[ListPlot[data, PlotStyle -> PointSize -> Medium],
Plot[nlm[x], {x, 0, 700}, PlotRange -> Full], PlotRange -> All]


Clearly, the obtained fit is much worse.

My questions are:

1. What is the objective function in NonlinearModelFit when introducing weights? The how-to says: "It is important to note that weights do not change the fitting or error estimates", but obviously the fit is changed. Thus, the weights have to be a part of the objective functions, which is minimized.

2. How can I obtain a better fit with introduced weights? I was playing around with the NMinimize options "NelderMead", "RandomSearch", "SimulatedAnnealing" and "DifferentialEvolution", but up to this point none of them was able to reproduce the fit obtained by ignoring the measurement errors.

-
isn't mathematica doing the right thing? The error on the first points are gigantic so it just ignores them... – chris May 22 '13 at 9:04
It also occurs with smaller errors. I edited my question, thanks. – Frederik Ziebell May 22 '13 at 9:19
The weights you choose for the last points are 10^8 times the one you use for the first point. What else would you expect than that the first point is almost completely ignored? – Sjoerd C. de Vries May 22 '13 at 11:22
I would expect that the residual on the first point is smaller than or at least close to the error. Since the measurement of a big quantity will yeald greater errors than a small quantity, the fit should account for this effect. – Frederik Ziebell May 22 '13 at 11:35

The Weights option operates on a point-by-point basis to say how important that individual point is to the complete fit. Let's take some simple data that obviously don't fit on a line and try to fit them with a line. With equal weighting, you get a compromise:

data = {{1, 1}, {2, 2.5}, {3, 3}};
nlm = NonlinearModelFit[data, a x + b, {a, b}, x, Weights -> {1, 1, 1}]
Show[ListPlot[data], Plot[nlm[x], {x, 0, 7}]]


If you weight the second point a lot (in comparison to the other two then it moves the fit closer to the second point.

nlm2 = NonlinearModelFit[data, a x + b, {a, b}, x, Weights -> {1, 100, 1}]
Show[ListPlot[data], Plot[nlm2[x], {x, 0, 7}]]


while if you weight the 1st and 3rd more heavily, it almost ignores the second completely:

nlm3 = NonlinearModelFit[data, a x + b, {a, b}, x, Weights -> {10, 1, 10}]
Show[ListPlot[data], Plot[nlm3[x], {x, 0, 7}]]


So -- what is it actually doing? You'd have to ask someone from Wolfram to know for sure, but it seems highly likely it's something like this:

Minimize ( $w_1*(f[x]-p_1)^2 + w_2*(f[x]-p_2)^2 + w_3*(f[x]-p_3)^2 )$

where the $p_i$ are the points and the $f[x]$ is your function. The weights are then proportional to the values you give in the Weights option. For instance, by setting $w_2=100$, you have made the fit to point $p_2$ much more important as it chooses the parameters. Now obviously, this example is for a simple linear fit (here you don't need the full power of NonlinearModelFit but it would be surprising if it operated fundamentally differently for different kinds of equation.

-
I also think that the objective function is something like the above-mentioned. Maybe even the "StandardizedResiduals", see reference.wolfram.com/mathematica/tutorial/… – Frederik Ziebell May 22 '13 at 10:32