2
$\begingroup$

I want to evaluate the goodness of two sets or more fitting parameters, using Rsquared and RMSE (root mean square error), Then how to code?)

 data = {
         {43.68, 1.}, {206.42, 0.76}, {398.11, 0.58},
         {1019.47, 0.42}, {1910.44, 0.33}, {2964.39, 0.26}, 
         {4116.84, 0.22}, {5318.17, 0.19}, {6505.9, 0.16}, 
         {7709.48, 0.15}, {8827.55, 0.13}, {9984.18, 0.12}, 
         {11015.08, 0.11}};

model = Sum[4/((2*k + 1)*Pi)*1(*c0*)* MittagLefflerE[β, ((-d)* (2*k + 1)^2*π^2*t^β)]*
Sin[(2*k + 1)*Pi*1/2], {k, 0, 10}];

M = NonlinearModelFit[data, model, {{β, 0.7}, {d, 0.0002}}, t];

Show[{ListPlot[data, PlotStyle -> Black],  Plot[M // Normal, {t, 
data[[1, 1]], data[[-1, 1]]},PlotStyle -> Red, PlotRange -> All]}]

In the above case, I use the fitting parameter set {β=0.7,d=0.0002} and get the fitting curve shown below.

{β=0.7,d=0.0002} fitting

In another case, I use the fitting parameter set {β=0.71,d=0.0029} and get the fitting curve shown below.

{β=0.71,d=0.0029} fitting

The two curves are quite similar, I need to use quantity criteria like "R-squared" and RMSE to assess the goodness of fitness. I searched in the documentation, and it seems no RMSE property is available.

How can I get the "RMSE" and "R-squared" from my FittedModel ?

$\endgroup$
16
  • 1
    $\begingroup$ RootMeanSquare is built-in. In Application section an example calculating RMSE of fitting result is shown. $\endgroup$
    – xzczd
    May 24 at 14:28
  • 1
    $\begingroup$ Your change of the initial estimates for the parameters does not appreciably change the "BestFitParameters" for the model {β -> 0.59542, d -> 0.00204182} $\endgroup$
    – Bob Hanlon
    May 24 at 14:36
  • 4
    $\begingroup$ As shown in the documentation, M[“BestFitParameters”] $\endgroup$
    – Bob Hanlon
    May 24 at 14:46
  • 2
    $\begingroup$ As mentioned above, RootMeanSquare can be used to calculate RMSE, and an example can be found in Application section of document, please check it carefully. Also, by searching RMSE in this site, one can find more examples: mathematica.stackexchange.com/search?q=rmse $\endgroup$
    – xzczd
    May 24 at 15:16
  • 3
    $\begingroup$ You might consider using $AIC_c$ to determine how many terms are needed in your model. Currently you have 11 terms ({k, 0, 10}) but using $AIC_c$ suggests that just 2 terms is more appropriate. In other words, using 11 terms is overfitting. $\endgroup$
    – JimB
    May 24 at 16:40

1 Answer 1

5
$\begingroup$

As mentioned in this page, definition of Root Mean Square Error (RMSE) is:

enter image description here

where

$f$ = forecasts (expected values or unknown results)

$o$ = observed values (known results)

$Σ$ = summation (“add up”)

$({{z_f}_i} – {{z_o}_i})^2$ = differences, squared

$N$ = sample size.

Now that you have your FittedModel M given by NonlinearModelFit, you can extract the properties like this

M["BestFitParameters"]
(* {β->0.59542,d->0.00204182} *)

It's obvious ${{z_f}_i} – {{z_o}_i}$ stands for residuals. Therefore, if you want the RMSE, you do

RootMeanSquare@M["FitResiduals"]

Also, the "RSquared" property is described in the documentation

M["RSquared"]

Also of interest

M["EstimatedVariance"]

And to assess the goodness of fit I recommend looking into the "AIC" property that provides the Akaike Information Criterion

$\endgroup$
7
  • 1
    $\begingroup$ StandardDeviation@M["FitResiduals"] isn't correct. Check the formula Barnston, 1992 on that page. See also my comments above. $\endgroup$
    – xzczd
    May 24 at 15:14
  • $\begingroup$ @xzczd I don't see it, if mine is wrong, please offer the correct explanation. $\endgroup$
    – rhermans
    May 24 at 22:17
  • $\begingroup$ Well, just search 1992 on that web page by e.g. pressing Ctrl+F, you'll see the formula is: i.stack.imgur.com/59ocd.png But definition of StandardDeviation is Sqrt@Total[(list-Mean[list])^2]/(Length[list]-1). $\endgroup$
    – xzczd
    May 25 at 0:39
  • 1
    $\begingroup$ …It's not merely $N$ v.s. $N-1$, notice the difference in the formula is calculated by $z_{f_i}-z_{o_i}$ where $z_{o_i}$ is the observed value, not the mean value. $\endgroup$
    – xzczd
    May 25 at 9:30
  • 2
    $\begingroup$ OK, done. Hopefully it's a bit clearer. $\endgroup$
    – xzczd
    May 25 at 14:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.