4
$\begingroup$

Fitting the following two time-series (two repeats of the same exponential growth process)

ts1 = {{682, 98}, {739, 165}, {784, 286}, {826, 470}, {850, 618}, {871, 779}}
ts2 = {{683, 92}, {739, 174}, {785, 299}, {827, 489}, {851, 637}, {871, 807}}

with

NonlinearModelFit[ts, a Exp[b t], {a, b}, t, Method -> "NMinimize"]

produces two very different results:

enter image description here

I am at a loss to explain what's gone wrong. "NMinimize" has always worked very well, but I wonder if there are other precautions I should always take with NonlinearModelFit.

$\endgroup$
1
  • $\begingroup$ Providing a ballpark initial guess works for me: NonlinearModelFit[ts, a Exp[b t], {a, {b, 1/800}}, t]. Doesn't work with Method -> "NMinimize" though, which ignores it. $\endgroup$
    – user484
    Nov 21, 2014 at 7:29

1 Answer 1

7
$\begingroup$

NMinimize[] with the Automatic method works nice ... sometimes.

You can help it by providing a better choice. Like in:

ts1 = {{682, 98}, {739, 165}, {784, 286}, {826, 470}, {850, 618}, {871, 779}};
ts2 = {{683, 92}, {739, 174}, {785, 299}, {827, 489}, {851, 637}, {871, 807}};
tt = {ts1, ts2};

nlm = NonlinearModelFit[#, a Exp[b t];, {a, b}, t, 
     Method -> {"NMinimize", {Method -> "SimulatedAnnealing"}}] & /@ tt;

nlm[[#]]["BestFitParameters"] & /@ {1, 2}
(* {{a -> 0.0368246, b -> 0.0114384}, {a -> 0.0348901, b -> 0.0115362}} *)

The default was probably using "DifferentialEvolution", as you can see here:

nlm = NonlinearModelFit[#,  a Exp[b t], {a, b}, t, Method -> {"NMinimize"}] & /@  tt;
nlm[[#]]["BestFitParameters"] & /@ {1, 2}
NMinimize[Total[(a Exp[b #[[1]]] - #[[2]])^2 & /@ #], {a, b}, 
                 Method -> {"DifferentialEvolution"}] & /@ tt

(* {{a -> 9.60194, b -> 0.00477938}, {a -> 0.03489, b -> 0.0115362}}
   {{95044.8, {a -> 9.60194, b -> 0.00477938}},
    {27.3624, {a -> 0.0348901, b -> 0.0115362}}}
*)

But the problem is that the function is very flat in the region of interest:

GraphicsRow[
 ContourPlot[Total[(a Exp[b #[[1]]] - #[[2]])^2 & /@ #], {a, 0, 10}, {b, 0, .01}, 
            AspectRatio -> 1] & /@ tt]

Mathematica graphics

So, this is what is happening with the convergence:

ptsk = Reap[NMinimize[Total[(a Exp[b #[[1]]] - #[[2]])^2 & /@ #], {a, b}, 
                      Method -> {"DifferentialEvolution"}, 
                      StepMonitor :> Sow[{a, b}]]][[2, 1]] & /@ tt;

GraphicsRow[ContourPlot[
    Total[(a Exp[b #[[1]]] - #[[2]])^2 & /@ #[[1]]], {a, 0, 14}, {b, 0, .02}, AspectRatio -> 1, 
     Epilog -> {Red, Arrow /@ Partition[#[[2]], 2, 1]}] & /@ Transpose[{tt, ptsk}]]

Mathematica graphics

Possibly a very small bump in a very flat surface is causing a large difference.

A more detailed view, where you can see the "return point" for the convergence:

Mathematica graphics

$\endgroup$
5
  • $\begingroup$ Yoohoo. Anyone listening to my comment here? If you linearize it makes for an easier problem to solve. In this example one gets a good result from nlml = LinearModelFit[Map[{#[[1]], Log[#[[2]]]} &, #], {1, t}, t] & /@ tt; funx = Map[Exp[#[[1]]]*Exp[t*#[[2]]] &, nlml[[#]]["BestFitParameters"] & /@ {1, 2}]. $\endgroup$ Nov 21, 2014 at 16:37
  • $\begingroup$ @DanielLichtblau Yup, but that takes out all the fun :) $\endgroup$ Nov 21, 2014 at 16:40
  • $\begingroup$ @DanielLichtblau To be clearer: the question asked "why this happens".I tried to answer that. $\endgroup$ Nov 21, 2014 at 16:43
  • 1
    $\begingroup$ Ignore the false indignation, I voted up your answer before I posted the silly comment. You did in fact explain the lame result. I just thought it funny that an example would come along so soon to justify the claim I had made in the prior comment. $\endgroup$ Nov 21, 2014 at 16:46
  • $\begingroup$ @DanielLichtblau All's well. Your comment is the way to go with the exponential model. As an aside, many years ago, while trying to model a relaxation process I found that sums of exponentials can be a pain in the. $\endgroup$ Nov 21, 2014 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.