0
$\begingroup$

Let us consider the function $f(x)$ which has the expansion in fractional powers of $x$ near $x=0$ as shown below.

p={5/6, 1, 5/3, 11/6, 2, 5/2, 8/3, 17/6, 3, 19/6, 10/3, 7/2, 11/3, 23/6, 4};
f[x_]:=Sum[c[i]*x^p[[i]],{i,1,15}];

I have a large set of numerical data (400 points) of the form $(x_k,f(x_k))$ in the interval $x\in(0,0.2)$ with the accuracy $err=10^{-5}$. The goal is to determine coefficients $c_i$ of the fit as accurately as possible.

One would expect that the accuracy of coefficients $c_i$ should be of the order $err/(0.2)^{p(i)}$. Say, the coefficient at $x^2$ should be fixed up to the error $10^{-5}/(0.2)^2\approx 0.00025$. However, I derived two approximations which both fit the data and the difference in their coefficients is much higher than that.

The difference of two fits gives the function

g[x_]:=0.0000161389 x^(5/6) - 0.0000602700 x + 0.00827206 x^(5/3) - 
 0.0363248 x^(11/6) + 0.0495515 x^2 - 0.133682 x^(5/2) + 
 0.0417227 x^(8/3) + 0.406065 x^(17/6) - 0.473090 x^3 + 
 0.0123648 x^(19/6) + 0.126177 x^(10/3) + 0.0891960 x^(7/2) - 
 0.279588 x^(11/3) + 0.449904 x^(23/6) - 0.261305 x^4

Its maximal value in the interval $x\in(0,0.2)$ is $2.6\times 10^{-5}$ is close to the error tolerance in the data. However, the coefficients are much higher than the above error estimate. I.e. this function $g(x)$ is very close to zero in the whole interval, but the coefficients in its expansion are much higher than the expected errors. This completely devalues the fit for all coefficients except the lowest ones.

How can I possibly get an accurate fit then?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ How did you derive these two approximations? Also, this sounds more like a general statistics problem than a Mathematica-specific one, to be honest. $\endgroup$
    – Sjoerd Smit
    Apr 26 at 13:33
  • $\begingroup$ It looks like the fit coefficients are highly correlated and therefore ill determined. I agree with @SjoerdSmit that this is a typical statistics problem and has nothing to do with Mathematica. $\endgroup$
    – Roman
    Apr 26 at 14:58

1 Answer 1

Reset to default
2
$\begingroup$

You have too many basis functions and therefore you end up with strongly correlated fit coefficients.

I'll try to construct an example to show what is happening.

Let's assemble some demo data:

data = Table[{x, x^4/100 + 10^-6 RandomVariate[NormalDistribution[]]},
             {x, 0, 1/5, 1/2000}];
ListPlot[data]

enter image description here

Now fit these data with the basis functions you provide:

p = {5/6, 1, 5/3, 11/6, 2, 5/2, 8/3, 17/6, 3, 19/6, 10/3, 7/2, 11/3, 23/6, 4};
fit = LinearModelFit[data, x^p, x, IncludeConstantBasis -> False];
fit["ParameterTable"]

$$ \begin{array}{l|llll} \text{} & \text{Estimate} & \text{Standard Error} & \text{t-Statistic} & \text{P-Value} \\ \hline x^{5/6} & -1.27552 & 3.40025 & -0.375125 & 0.707774 \\ x & 9.61197 & 25.1332 & 0.382441 & 0.702345 \\ x^{5/3} & -19037.2 & 47842.1 & -0.397918 & 0.69091 \\ x^{11/6} & 169800. & 425903. & 0.398681 & 0.690349 \\ x^2 & -496734. & 1.24674\times 10^6 & -0.398425 & 0.690537 \\ x^{5/2} & 3.242\times 10^7 & 8.25662\times 10^7 & 0.392655 & 0.694791 \\ x^{8/3} & -2.1707\times 10^8 & 5.57458\times 10^8 & -0.389393 & 0.6972 \\ x^{17/6} & 7.45553\times 10^8 & 1.93348\times 10^9 & 0.385603 & 0.700003 \\ x^3 & -1.64553\times 10^9 & 4.31499\times 10^9 & -0.381353 & 0.703151 \\ x^{19/6} & 2.50338\times 10^9 & 6.64542\times 10^9 & 0.376707 & 0.706598 \\ x^{10/3} & -2.67567\times 10^9 & 7.19809\times 10^9 & -0.371719 & 0.710306 \\ x^{7/2} & 1.98576\times 10^9 & 5.41912\times 10^9 & 0.366436 & 0.714241 \\ x^{11/3} & -9.78551\times 10^8 & 2.71142\times 10^9 & -0.360899 & 0.718372 \\ x^{23/6} & 2.88751\times 10^8 & 8.13048\times 10^8 & 0.355146 & 0.722674 \\ x^4 & -3.86909\times 10^7 & 1.10796\times 10^8 & -0.349207 & 0.727124 \\ \end{array} $$

Here already we see that the Standard Errors of the coefficients are very large, larger than the coefficients themselves. The problem becomes even more apparent when we check how these fit coefficients are correlated:

fit["CorrelationMatrix"] // MatrixForm

$$ \left( \begin{array}{ccccccccccccccc} 1. & -0.999719 & 0.993488 & -0.990828 & 0.9878 & -0.976859 & 0.972701 & -0.968338 & 0.963795 & -0.959096 & 0.954265 & -0.949322 & 0.944285 & -0.939171 & 0.933995 \\ -0.999719 & 1. & -0.995896 & 0.993729 & -0.991176 & 0.981551 & -0.977795 & 0.973815 & -0.969638 & 0.965289 & -0.96079 & 0.956162 & -0.951425 & 0.946597 & -0.941692 \\ 0.993488 & -0.995896 & 1. & -0.999768 & 0.999093 & -0.994731 & 0.992602 & -0.990185 & 0.987507 & -0.984593 & 0.981469 & -0.978156 & 0.974675 & -0.971046 & 0.967287 \\ -0.990828 & 0.993729 & -0.999768 & 1. & -0.999778 & 0.996701 & -0.994974 & 0.992945 & -0.99064 & 0.988087 & -0.985309 & 0.982329 & -0.979168 & 0.975846 & -0.972381 \\ 0.9878 & -0.991176 & 0.999093 & -0.999778 & 1. & -0.998186 & 0.996856 & -0.99521 & 0.993276 & -0.991081 & 0.988647 & -0.985999 & 0.983157 & -0.980142 & 0.976971 \\ -0.976859 & 0.981551 & -0.994731 & 0.996701 & -0.998186 & 1. & -0.999817 & 0.999286 & -0.998433 & 0.997285 & -0.995864 & 0.994193 & -0.992296 & 0.99019 & -0.987897 \\ 0.972701 & -0.977795 & 0.992602 & -0.994974 & 0.996856 & -0.999817 & 1. & -0.999826 & 0.99932 & -0.998509 & 0.997415 & -0.996062 & 0.994472 & -0.992664 & 0.990657 \\ -0.968338 & 0.973815 & -0.990185 & 0.992945 & -0.99521 & 0.999286 & -0.999826 & 1. & -0.999834 & 0.999353 & -0.998581 & 0.99754 & -0.996252 & 0.994736 & -0.993013 \\ 0.963795 & -0.969638 & 0.987507 & -0.99064 & 0.993276 & -0.998433 & 0.99932 & -0.999834 & 1. & -0.999842 & 0.999384 & -0.998649 & 0.997658 & -0.996431 & 0.994987 \\ -0.959096 & 0.965289 & -0.984593 & 0.988087 & -0.991081 & 0.997285 & -0.998509 & 0.999353 & -0.999842 & 1. & -0.99985 & 0.999414 & -0.998714 & 0.99777 & -0.996601 \\ 0.954265 & -0.96079 & 0.981469 & -0.985309 & 0.988647 & -0.995864 & 0.997415 & -0.998581 & 0.999384 & -0.99985 & 1. & -0.999857 & 0.999442 & -0.998776 & 0.997876 \\ -0.949322 & 0.956162 & -0.978156 & 0.982329 & -0.985999 & 0.994193 & -0.996062 & 0.99754 & -0.998649 & 0.999414 & -0.999857 & 1. & -0.999864 & 0.999469 & -0.998834 \\ 0.944285 & -0.951425 & 0.974675 & -0.979168 & 0.983157 & -0.992296 & 0.994472 & -0.996252 & 0.997658 & -0.998714 & 0.999442 & -0.999864 & 1. & -0.99987 & 0.999494 \\ -0.939171 & 0.946597 & -0.971046 & 0.975846 & -0.980142 & 0.99019 & -0.992664 & 0.994736 & -0.996431 & 0.99777 & -0.998776 & 0.999469 & -0.99987 & 1. & -0.999876 \\ 0.933995 & -0.941692 & 0.967287 & -0.972381 & 0.976971 & -0.987897 & 0.990657 & -0.993013 & 0.994987 & -0.996601 & 0.997876 & -0.998834 & 0.999494 & -0.999876 & 1. \\ \end{array} \right) $$

Some coefficients, for example #1 and #3, are almost 100% correlated!

Improve this answer
$\endgroup$
1
  • $\begingroup$ +1 The predictions of the response will be as expected (although the calculation of prediction and confidence intervals will be affected by precision issues in this case). It's just that one can't estimate the individual coefficients very well when the model is overparameterized. $\endgroup$
    – JimB
    Apr 26 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.