# Goodness of fit [closed]

I want to evaluate the goodness of two sets or more fitting parameters, using Rsquared and RMSE (root mean square error), Then how to code?)

 data = {
{43.68, 1.}, {206.42, 0.76}, {398.11, 0.58},
{1019.47, 0.42}, {1910.44, 0.33}, {2964.39, 0.26},
{4116.84, 0.22}, {5318.17, 0.19}, {6505.9, 0.16},
{7709.48, 0.15}, {8827.55, 0.13}, {9984.18, 0.12},
{11015.08, 0.11}};

model = Sum[4/((2*k + 1)*Pi)*1(*c0*)* MittagLefflerE[β, ((-d)* (2*k + 1)^2*π^2*t^β)]*
Sin[(2*k + 1)*Pi*1/2], {k, 0, 10}];

M = NonlinearModelFit[data, model, {{β, 0.7}, {d, 0.0002}}, t];

Show[{ListPlot[data, PlotStyle -> Black],  Plot[M // Normal, {t,
data[[1, 1]], data[[-1, 1]]},PlotStyle -> Red, PlotRange -> All]}]


In the above case, I use the fitting parameter set {β=0.7,d=0.0002} and get the fitting curve shown below.

In another case, I use the fitting parameter set {β=0.71,d=0.0029} and get the fitting curve shown below.

The two curves are quite similar, I need to use quantity criteria like "R-squared" and RMSE to assess the goodness of fitness. I searched in the documentation, and it seems no RMSE property is available.

How can I get the "RMSE" and "R-squared" from my FittedModel ?

• RootMeanSquare is built-in. In Application section an example calculating RMSE of fitting result is shown. May 24 at 14:28
• Your change of the initial estimates for the parameters does not appreciably change the "BestFitParameters" for the model {β -> 0.59542, d -> 0.00204182} May 24 at 14:36
• As shown in the documentation, M[“BestFitParameters”] May 24 at 14:46
• As mentioned above, RootMeanSquare can be used to calculate RMSE, and an example can be found in Application section of document, please check it carefully. Also, by searching RMSE in this site, one can find more examples: mathematica.stackexchange.com/search?q=rmse May 24 at 15:16
• You might consider using $AIC_c$ to determine how many terms are needed in your model. Currently you have 11 terms ({k, 0, 10}) but using $AIC_c$ suggests that just 2 terms is more appropriate. In other words, using 11 terms is overfitting.
– JimB
May 24 at 16:40

As mentioned in this page, definition of Root Mean Square Error (RMSE) is:

where

$$f$$ = forecasts (expected values or unknown results)

$$o$$ = observed values (known results)

$$Σ$$ = summation (“add up”)

$$({{z_f}_i} – {{z_o}_i})^2$$ = differences, squared

$$N$$ = sample size.

Now that you have your FittedModel M given by NonlinearModelFit, you can extract the properties like this

M["BestFitParameters"]
(* {β->0.59542,d->0.00204182} *)


It's obvious $${{z_f}_i} – {{z_o}_i}$$ stands for residuals. Therefore, if you want the RMSE, you do

RootMeanSquare@M["FitResiduals"]


Also, the "RSquared" property is described in the documentation

M["RSquared"]


Also of interest

M["EstimatedVariance"]


And to assess the goodness of fit I recommend looking into the "AIC" property that provides the Akaike Information Criterion

• StandardDeviation@M["FitResiduals"] isn't correct. Check the formula Barnston, 1992 on that page. See also my comments above. May 24 at 15:14
• @xzczd I don't see it, if mine is wrong, please offer the correct explanation. May 24 at 22:17
• Well, just search 1992 on that web page by e.g. pressing Ctrl+F, you'll see the formula is: i.stack.imgur.com/59ocd.png But definition of StandardDeviation is Sqrt@Total[(list-Mean[list])^2]/(Length[list]-1). May 25 at 0:39
• …It's not merely $N$ v.s. $N-1$, notice the difference in the formula is calculated by $z_{f_i}-z_{o_i}$ where $z_{o_i}$ is the observed value, not the mean value. May 25 at 9:30
• OK, done. Hopefully it's a bit clearer. May 25 at 14:19