3
$\begingroup$

Assume we have some data of measurements

SeedRandom[9];
data = Table[{i,2*(1 + 5*RandomReal[])*Exp[-0.01*(1 + RandomReal[])*i]}, {i, {0, 100, 200, 400, 700}}];

and measurement errors associated to it

errors = 10^# & /@ Range[0, -4, -1];

Ignoring the weights in a first step, the fitting works very well:

nlm = NonlinearModelFit[data, a*Exp[-b*x], {a, b}, x, Method -> "NMinimize"];
Show[ListPlot[data, PlotStyle -> PointSize -> Medium], 
Plot[nlm[x], {x, 0, 700}, PlotRange -> Full], PlotRange -> All]

Fit without measurement errors

Now, according to this how-to, we can include the measurement errors to the fitting procedure:

nlm = NonlinearModelFit[data, a*Exp[-b*x], {a, b}, x, Weights -> 1/errors^2, VarianceEstimatorFunction -> (1 &), Method -> "NMinimize"];
Show[ListPlot[data, PlotStyle -> PointSize -> Medium], 
Plot[nlm[x], {x, 0, 700}, PlotRange -> Full], PlotRange -> All]

Clearly, the obtained fit is much worse.

Fit with measurement errors

My questions are:

  1. What is the objective function in NonlinearModelFit when introducing weights? The how-to says: "It is important to note that weights do not change the fitting or error estimates", but obviously the fit is changed. Thus, the weights have to be a part of the objective functions, which is minimized.

  2. How can I obtain a better fit with introduced weights? I was playing around with the NMinimize options "NelderMead", "RandomSearch", "SimulatedAnnealing" and "DifferentialEvolution", but up to this point none of them was able to reproduce the fit obtained by ignoring the measurement errors.

$\endgroup$
4
  • 1
    $\begingroup$ isn't mathematica doing the right thing? The error on the first points are gigantic so it just ignores them... $\endgroup$
    – chris
    May 22, 2013 at 9:04
  • $\begingroup$ It also occurs with smaller errors. I edited my question, thanks. $\endgroup$ May 22, 2013 at 9:19
  • $\begingroup$ The weights you choose for the last points are 10^8 times the one you use for the first point. What else would you expect than that the first point is almost completely ignored? $\endgroup$ May 22, 2013 at 11:22
  • $\begingroup$ I would expect that the residual on the first point is smaller than or at least close to the error. Since the measurement of a big quantity will yeald greater errors than a small quantity, the fit should account for this effect. $\endgroup$ May 22, 2013 at 11:35

1 Answer 1

5
$\begingroup$

The Weights option operates on a point-by-point basis to say how important that individual point is to the complete fit. Let's take some simple data that obviously don't fit on a line and try to fit them with a line. With equal weighting, you get a compromise:

data = {{1, 1}, {2, 2.5}, {3, 3}};
nlm = NonlinearModelFit[data, a x + b, {a, b}, x, Weights -> {1, 1, 1}]
Show[ListPlot[data], Plot[nlm[x], {x, 0, 7}]]

If you weight the second point a lot (in comparison to the other two then it moves the fit closer to the second point.

nlm2 = NonlinearModelFit[data, a x + b, {a, b}, x, Weights -> {1, 100, 1}]
Show[ListPlot[data], Plot[nlm2[x], {x, 0, 7}]]

while if you weight the 1st and 3rd more heavily, it almost ignores the second completely:

nlm3 = NonlinearModelFit[data, a x + b, {a, b}, x, Weights -> {10, 1, 10}]
Show[ListPlot[data], Plot[nlm3[x], {x, 0, 7}]]

enter image description here

So -- what is it actually doing? You'd have to ask someone from Wolfram to know for sure, but it seems highly likely it's something like this:

Minimize ( $w_1*(f[x]-p_1)^2 + w_2*(f[x]-p_2)^2 + w_3*(f[x]-p_3)^2 )$

where the $p_i$ are the points and the $f[x]$ is your function. The weights are then proportional to the values you give in the Weights option. For instance, by setting $w_2=100$, you have made the fit to point $p_2$ much more important as it chooses the parameters. Now obviously, this example is for a simple linear fit (here you don't need the full power of NonlinearModelFit but it would be surprising if it operated fundamentally differently for different kinds of equation.

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.