When performing a fit using NonlinearModelFit (or any of Mathematica fitting functions, but I am particularly interested in nonlinear models), Mathematica makes a covariance matrix available for all the fit parameters.
I was wondering if anyone has an idea of how Mathematica does this error estimation on parameters?
According to Mathematica documentation and my own calculations direct answer is:
errorList = Sqrt[Diagonal[cov]]
where cov is covariance matrix, and errorList list of parameter uncertainties in given order. Next, cov in Mathematica is accessible as:
cov=nlm["CovarianceMatrix"]
or it can be estimated from Hessian matrix
$H_{i,j}=\frac{\partial^2 RSS}{\partial x_i\partial x_j},$
where $RSS$ is residual sum of squares at best fit point and $x_i$ are fit parameters. Finally,
$\mathrm{cov}=\frac{2 RSS }{n-k}H^{-1},$
where $n$ is number of points and $k$ number of fit parameters. Full expression:
$\mathrm{errorList}=\sqrt{\frac{2 RSS}{(n-k)}\mathrm{Diagonal[}H^{-1}\mathrm{]}}.$
The statistics are computed from a linear approximation around the best fit. It is described in the Documentation Center -> Virtual Book under "Statistical Model Analysis" in the subsection on "Nonlinear Models".
