5
$\begingroup$

I'm not quite sure, whether this question is one for this forum. I have problems to interpret the standard errors and p-values for my fit parameters. They are so small, that I don't know, whether I did use the fit in a proper way or not. The parameter values itself are good, they reproduce results from a colleague...
...but let's start from the beginning:

I have a set of real measurement data

data = {{2.15838, 4.77993*10^-7}, {2.18534, 4.19629*10^-7}, {2.21001, 
4.09502*10^-7}, {2.25854, 3.99853*10^-7}, {2.2836, 
3.84017*10^-7}, {2.30747, 3.70902*10^-7}, {2.33081, 
3.59127*10^-7}, {2.35137, 3.50775*10^-7}, {2.37643, 
3.3436*10^-7}, {2.39882, 3.2283*10^-7}, {2.42257, 
3.01332*10^-7}, {2.4461, 2.98024*10^-7}, {2.4732, 
2.89137*10^-7}, {2.49911, 2.82813*10^-7}, {2.52499, 
2.68812*10^-7}, {2.55184, 2.61058*10^-7}, {2.58006, 
2.2438*10^-7}, {2.64673, 2.30085*10^-7}, {2.71751, 
2.05682*10^-7}, {2.78639, 1.92553*10^-7}, {2.88777, 
1.75161*10^-7}, {2.96864, 1.61607*10^-7}, {3.051, 
1.49176*10^-7}, {3.16198, 1.25115*10^-7}, {3.33296, 
1.20133*10^-7}, {3.43768, 1.01685*10^-7}, {3.56674, 
9.66192*10^-8}, {3.68825, 7.75714*10^-8}, {3.79472, 
6.92602*10^-8}, {3.87449, 6.89086*10^-8}, {3.95223, 
6.43365*10^-8}, {4.08792, 5.56352*10^-8}, {4.14613, 
5.2924*10^-8}, {4.21338, 4.94536*10^-8}, {4.24873, 
4.87433*10^-8}, {4.25848, 4.78836*10^-8}, {4.26615, 
4.7465*10^-8}, {4.31084, 4.81383*10^-8}, {4.44111, 
4.64897*10^-8}, {4.52387, 4.56303*10^-8}, {4.56865, 
4.51747*10^-8}, {4.6603, 4.43627*10^-8}, {4.74556, 
4.33014*10^-8}, {4.82304, 4.2891*10^-8}, {4.92704, 
4.18981*10^-8}, {5.0293, 4.11988*10^-8}, {5.09724, 
4.08886*10^-8}, {5.20107, 4.02819*10^-8}, {5.27865, 
4.11364*10^-8}, {5.35455, 4.10741*10^-8}, {5.49134, 
4.0462*10^-8}, {5.62088, 3.70902*10^-8}, {5.68181, 
3.67886*10^-8}, {5.77189, 3.65903*10^-8}, {5.8566, 
3.6443*10^-8}, {5.97267, 3.61518*10^-8}, {6.02845, 
3.63941*10^-8}, {6.08799, 3.6008*10^-8}, {6.24226, 
3.58653*10^-8}, {6.35647, 3.56767*10^-8}, {6.43191, 
3.56767*10^-8}, {6.51399, 3.55366*10^-8}, {6.5876, 
3.54902*10^-8}};

with estimated systematic uncertainties of 10%

yerrors = {4.77993*10^-8, 4.19629*10^-8, 4.09502*10^-8, 3.99853*10^-8,
3.84017*10^-8, 3.70902*10^-8, 3.59127*10^-8, 3.50775*10^-8, 
3.3436*10^-8, 3.2283*10^-8, 3.01332*10^-8, 2.98024*10^-8, 
2.89137*10^-8, 2.82813*10^-8, 2.68812*10^-8, 2.61058*10^-8, 
2.2438*10^-8, 2.30085*10^-8, 2.05682*10^-8, 1.92553*10^-8, 
1.75161*10^-8, 1.61607*10^-8, 1.49176*10^-8, 1.25115*10^-8, 
1.20133*10^-8, 1.01685*10^-8, 9.66192*10^-9, 7.75714*10^-9, 
6.92602*10^-9, 6.89086*10^-9, 6.43365*10^-9, 5.56352*10^-9, 
5.2924*10^-9, 4.94536*10^-9, 4.87433*10^-9, 4.78836*10^-9, 
4.7465*10^-9, 4.81383*10^-9, 4.64897*10^-9, 4.56303*10^-9, 
4.51747*10^-9, 4.43627*10^-9, 4.33014*10^-9, 4.2891*10^-9, 
4.18981*10^-9, 4.11988*10^-9, 4.08886*10^-9, 4.02819*10^-9, 
4.11364*10^-9, 4.10741*10^-9, 4.0462*10^-9, 3.70902*10^-9, 
3.67886*10^-9, 3.65903*10^-9, 3.6443*10^-9, 3.61518*10^-9, 
3.63941*10^-9, 3.6008*10^-9, 3.58653*10^-9, 3.56767*10^-9, 
3.56767*10^-9, 3.55366*10^-9, 3.54902*10^-9};

And I use a NonlinearModelFit with the following function:

c1 = 1.3*10^9; c2 = 9.2;
fitfunc = ((p1*c1^2)/c2)*x*Exp[-p2*x] + p3

while the constants and parameters are physical values. Then I fit the data with weights and do get sensible values for the parameters

nlmFIT = NonlinearModelFit[data, fitfunc, {p1, p2, p3}, x, 
Weights -> 1/yerrors^2, VarianceEstimatorFunction -> (1 &)]

fitted data

But the interpretation of the goodness of the fit gives me a headache, since the errors and p-values given below are really, really small:

nlmFIT["ParameterTable"]

I read here in the forum and elsewhere, but I didn't find a good explanation (which I understood ;-)).
Any ideas and hints are welcome. Is it a result of the small numbers of my parameters? Or should I use other values for the interpretation of my results?

$\endgroup$
  • 1
    $\begingroup$ You are right to be suspicious of the standard errors of the parameters. If you would examine nlmFIT["CorrelationMatrix"] you'll find all of the correlations are estimated to be either 1 or -1 which also spells trouble. While the cause of such high magnitude correlations can be an overparameterized model, in this case the problem is that the parameters are on such wildly different scales. You'll get much better results if you use multiplicative factors to get the parameters more similar to each other: fitfunc = ((10^(-23)*p1*c1^2)/c2)*x*Exp[-p2*x] + p3*10^-8. $\endgroup$ – JimB May 7 '17 at 5:44
5
$\begingroup$

You are right to be suspicious of the standard errors of the parameters. If you would examine nlmFIT["CorrelationMatrix"] you'll find all of the correlations are estimated to be either 1 or -1 which also spells trouble. While the cause of such high magnitude correlations can be an overparameterized model, in this case the problem is that the parameters are on such wildly different scales.

First, consider fitting the model without weights but with multiplicative factors to get all of the parameters roughly on the same scale:

c1 = 1.3*10^9; c2 = 9.2;
fitfunc = ((10^(-23) p1*c1^2)/c2)*x*Exp[-p2*x] + 10^(-8) p3;
nlmFIT = NonlinearModelFit[data, fitfunc, {p1, p2, p3}, x];
nlmFIT["ParameterTable"]

Parameter table

Show the data and the resulting fit:

Show[Plot[nlmFIT[x], {x, 2, 7}, PlotRange -> All, AxesOrigin -> {2, 0},
  Frame -> True, FrameLabel -> {"x", "y"}],
 ListPlot[data, PlotStyle -> Black]]

Data and fit

So on a quick examination the fit is not outrageous. But do you need to add weights? Your estimated weights are by definition exactly proportional to the observed values. To see if that weighting might be reasonable we look at the residuals from the unweighted model fit vs the predicted values:

ListPlot[Transpose[{nlmFIT["PredictedResponse"], nlmFIT["FitResiduals"]}],
 Frame -> True, FrameLabel -> {"Predicted response", "Residual"}]

Residuals vs predicted

If the variability increases with increasing predicted values, we would see a "fan shape" for the scatter of points with low variability on the left of the graph and larger and larger amounts of variability as we move to the right of the graph. But we don't see that.

We also don't see a "nice" uniform scatter of points across the range of the predicted values. This can occur for a variety of reasons. One possibility is that there are other terms that should be in the model but aren't. Another (and this would be my guess as to what's going on) would be that the data points are taken across time (essentially the x variable) and there is serial correlation among the observations. Here's a plot of the residuals vs the x variable:

ListPlot[nlmFIT["FitResiduals"], Frame -> True, FrameLabel -> {"x", "Residual"}]

Residuals vs x

So in summary I don't think your proposed weights are appropriate for this data and model. However, there are still some "lack-of-fit" features that probably should be addressed. What to do about those would depend on how the data is generated (serial correlation over time?, instrument flucation over time? multiple instruments? observations taken in separate groups? etc.).

$\endgroup$
  • $\begingroup$ Very insightful. $\endgroup$ – Anton Antonov May 7 '17 at 15:44
  • $\begingroup$ Thanks a lot for the detailed answer. I already applied some re-normalization factors and for sure the weights have to be thought over again. Your guess is right concerning the data generation. The values are taken during some hours and the multiple instruments in the DAQ seem to show some time-dependent effects. $\endgroup$ – Lea May 8 '17 at 7:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.