How does Mma compute Confidence Intervals?

Question

I am trying to understand how Mma computes the Confidence Intervals after a NonlinearModelFit. Consider the following example:

data = {{0, 1}, {1, 0}, {3, 2}, {5, 4}, {6, 4}, {7, 5}};
nlm = NonlinearModelFit[data, Log[a + b x^2], {a, b}, x];
Show[ListPlot[data, Joined -> False], Plot[Normal@nlm, {x, 0, 8}],ImageSize -> 200]
(*out*)

Which comes with the following R squared and confidence intervals:

nlm["RSquared"]
nlm["ParameterConfidenceIntervalTable"]

(*Out*)0.957573
(*Out*)

The table above assumes the default 95% Confidence Interval and, I am assuming, normal distribution around a=1.50632 and b=1.42633.

Questions:

1. How does Mma estimate the Standard Error of a and b?
2. The mean used to estimate such confidence intervals is the sample mean, i.e. ~2.6667?

I could not find in the documentation how this was done from a fitting. Thank you in advance.

Answer 1

Standard errors and confidence intervals from linear and nonlinear regressions are obtained from the covariance matrix. Details about the covariance matrix can be obtained here.

Briefly, the square root of the diagonal elements of the covariance matrix gives us the standard errors:

se = Sqrt[nlm["CovarianceMatrix"]] // Diagonal
(* {1.10159, 0.600123} *)

The confidence interval is obtained by multiplying the standard error by the value of Student's t for the given confidence level and degrees of freedom. For this set of data, the degrees of freedom is 4 and for a confidence level of 95%

ci = InverseCDF[StudentTDistribution[4], 1 - 0.05/2]
(* 2.77645 *)

Thus, we obtain the last column of the ParameterConfidenceIntervalTable:

Transpose[{{a, b} - se ci /. nlm["BestFitParameters"], 
   {a, b} + se ci /. nlm["BestFitParameters"]}]
(* {{-1.55217, 4.56481}, {-0.239879, 3.09254}} *)

Note that this method assumes that the error in the parameters is normally distributed, which is probably not the case for a nonlinear model. I would be hesitant to rely heavily on the confidence intervals reported by Mathematica in light of this issue. If you have access to the following journals, this article presents a monte carlo method for deriving confidence intervals that doesn't require the assumption of normality and the jacknife method (I can't put my finger on the reference I use, but several options show up with a google search) is fairly straightforward to implement.