# Why does NMinimize not follow MaxIterations?

ClearAll[k1, k2, k3];
i = 0;
function[k1_?NumericQ, k2_?NumericQ, k3_?NumericQ] := Module[{result},
result = k1 + k2 + k3;
i = i + 1;
Return[result];
];

NMinimize[{function[k1, k2, k3],
100 > k1 > 10 && 1000 > k2 > 100 && 1000 > k3 > 10}, {k1, k2, k3},
MaxIterations -> 10]

Print[i]


Running this code should result in NMinimize running over MaxIterations -> 10 times, but, i=5528 times. Why?

This is an issue for a more complex problem, where it is actually important that NMinimize stops after $$n$$ iterations.

• Interestingly, increasing the value of MaxIterations will also change i
– Tomi
Commented Jan 15, 2021 at 17:33
• I suppose the function is evaluated multiple times at each step. But 500 per step seems excessive to me, but I’m not sure what to blame it on yet. Commented Jan 15, 2021 at 23:12
• Add StepMonitor :> (step[i] = {k1, k2, k3}) and examine DownValues[step] after the run. There are 10 or 11 steps where progress is made, followed by another 10 or so, where nothing changes, except function is called another few thousand times. Commented Jan 15, 2021 at 23:31

This happens because all NMinimize's global search methods have a post-processing part that does local search. This is documented, see: "Numerical Nonlinear Global Optimization".

Whether to use local search or not is controlled with the method sub-option "PostProcess". If we put "PostProcess" to False, then the max iterations specification is respected (and message of non-convergence might be given.)

Here is a demonstration:

ClearAll[k1, k2, k3];
function[k1_?NumericQ, k2_?NumericQ, k3_?NumericQ] :=
Module[{result},
result = k1 + k2 + k3;
i = i + 1;
result
];

i = 0;
NMinimize[{
function[k1, k2, k3],
100 > k1 > 10 && 1000 > k2 > 100 && 1000 > k3 > 10}, {k1, k2, k3},
MaxIterations -> 10, Method -> {"NelderMead", "PostProcess" -> False}]

(* During evaluation of In[33]:= NMinimize::cvmit: Failed to converge to the requested accuracy or precision within 10 iterations. *)

(* {120.425, {k1 -> 10.0201, k2 -> 100.308, k3 -> 10.0967}} *)

i

(* 20 *)


Here is a run of all methods:

Association@
Table[m -> (i = 0;
NMinimize[{function[k1, k2, k3],
100 > k1 > 10 && 1000 > k2 > 100 && 1000 > k3 > 10}, {k1, k2, k3},
MaxIterations -> 10, Method -> {m, "PostProcess" -> False}];
"RandomSearch", "SimulatedAnnealing"}}]

(*
During evaluation of In[48]:= NMinimize::cvmit: Failed to converge to the requested accuracy or precision within 10 iterations.

During evaluation of In[48]:= NMinimize::cvmit: Failed to converge to the requested accuracy or precision within 10 iterations.

During evaluation of In[48]:= NMinimize::cvmit: Failed to converge to the requested accuracy or precision within 10 iterations.

During evaluation of In[48]:= General::stop: Further output of NMinimize::cvmit will be suppressed during this calculation.
*)

(* <|"Automatic" -> 1104,
"DifferentialEvolution" -> 1104,
"RandomSearch" -> 46000,
"SimulatedAnnealing" -> 233|> *)


Maybe Return in the StepMonitor could be used:

ClearAll[k1, k2, k3, step, function];
i = 0;
function[k1_?NumericQ, k2_?NumericQ, k3_?NumericQ] :=
Module[{result},
result = k1 + k2 + k3;
i = i + 1;
result];

k = 0;
maxsteps = 10;
NMinimize[
{function[k1, k2, k3],
100 > k1 > 10 && 1000 > k2 > 100 && 1000 > k3 > 10},
{k1, k2, k3},
StepMonitor :> (
++k;
step[k, i] = {k1, k2, k3};  (* optional; for diagnostics below *)
If[k >= maxsteps,
Return[{function[k1, k2, k3],
NMinimize]])
]
i

(*
{120.22,
{HoldPattern[k1] -> 10.0397,
HoldPattern[k2] -> 100.095,
HoldPattern[k3] -> 10.085}}

995  <-- function calls
*)


The data is step is not particularly interesting but is merely proof-of-concept.

Length@DownValues@step
DownValues@step

(*  10  *)