1
$\begingroup$

If you examine the operation of NonlinearModelFit you find that the model is evaluated with the same parameters again and again. I have a very expensive model so this wastes much time.

This is the example from Help in version 12.1.

data = {{6.47, 3.65}, {7.43, 
    3.45}, {3.9, -2.94}, {4.8, -1.29}, {2.48, -0.35}, {6.32, 
    3.16}, {2.59, -1.19}, {9.13, -2.}, {3.81, -3.04}, {3.33, -2.68}};

ClearAll[model];
model[a_?NumberQ, b_?NumberQ, c_?NumberQ] :=  Module[{y, x},
  Sow[{a, b, c}]; 
  First[y /. 
    NDSolve[{y''[x] + a y[x] == 0, y[0] == b, y'[0] == c}, 
     y, {x, 0, 10}]]]

t1 = Timing[
  r1 = Reap[
     nlm1 = NonlinearModelFit[data, model[a, b, c][x], {a, b, c}, x, 
       Method -> "Gradient"]];]

(* {4.375, Null}  *)

I have added a Sow and Reap so we can find the number of evaluations. Looking at the number of terms that were reaped.

L1 = Length[r1[[2, 1]]]

(* 6280 *)

If we look at the first 100 values they all seem to be the same. However, they might be almost similar so I eliminate identical terms using Union

Union[r1[[2, 1, 1 ;; 100]]] // InputForm

(* {{1., 1., 1.}, {1., 1., 1.0000000149011612}, 
 {1., 1.0000000149011612, 1.}, {1.0000000149011612, 1., 1.}} *)

This might need a closer look because we are looking at small differences, however, it does seem that only 4 evaluations were different. If there is repeat evaluation then memoization would seem to be a good idea. I implement this as follows:

ClearAll[model];
model[a_?NumberQ, b_?NumberQ, c_?NumberQ] := 
 model[a, b, c] =  Module[{y, x},
   Sow[{a, b, c}]; 
   First[y /. 
     NDSolve[{y''[x] + a y[x] == 0, y[0] == b, y'[0] == c}, 
      y, {x, 0, 10}]]]

t2 = Timing[
  r2 = Reap[
     nlm2 = NonlinearModelFit[data, model[a, b, c][x], {a, b, c}, x, 
       Method -> "Gradient"]];]

(* {0.1875, Null} *)

The number of function evaluations is now

L2 = Length[r2[[2, 1]]]

(* 227 *)

Thus a much smaller number of function evaluations and the calculation runs much faster. The ratio of time and function evaluations is

{t1[[1]]/t2[[1]], L1/L2 // N}

(* {23.3333, 27.6652} *)

This is a speed up of times 20. So it seems that NonlinearModelFit is doing useless function evaluations and that memoization makes a huge difference.

My questions are:

  1. Am I missing something here?

  2. Why does NonlinearModelFit do useless function evaluations?

Thanks

$\endgroup$
  • $\begingroup$ I can't answer why the duplicate function evaluations but what would seem to be a bigger issue is that the fit underestimates all but 1 data point. The model does not appear to be what generates the data: Show[ListPlot[data], Plot[nlm1[x], {x, 0, 10}]]. $\endgroup$ – JimB Aug 23 at 1:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.