6
$\begingroup$

Why does NMinimize using NelderMead without PostProcessing always stop at the nearest decade of iterations, e.g. 100, 110, 120... Perhaps Catch Throw can prevent this e.g., see this answer, but that is not the question, which is, why this behavior; why does it stop at exact decades for iterations?

tesx = {1, 2, 3, 4, 5};
tesy = {4, 2, 1, 2, 4};
Do[it = 0;
 ous = NMinimize[Norm[tesy - a2 tesx^2 + a1 tesx + a0], {a2, a1, a0},
 WorkingPrecision -> wp, MaxIterations -> 20000, 
 Method -> {"NelderMead", "ShrinkRatio" -> 0.9, 
 "PostProcess" -> False}, StepMonitor :> {it++}];
Print[it, "\t", ous[[1]]], {wp, 19, 50}];

#iterations-------minimum

  120   0.2390457219044427099
  210   0.23904572186687872818
  120   0.239045721866879165778
  130   0.2390457218672158088172
  130   0.23904572186690571699009
  140   0.239045721866878786690472
  160   0.2390457218668787280408863
  160   0.23904572186687894375271261
  130   0.239045721866882808032508313
  130   0.2390457218668795190897850923
  150   0.23904572186687876995648613429
  140   0.239045721866878906036855231273
  170   0.2390457218668787279938937091596
  130   0.23904572186687873329328192813721
  160   0.239045721866878729494553896289954
  190   0.2390457218668787279937731318471242
  180   0.23904572186687872799432232212248446
  180   0.239045721866878728044197726176829170
  170   0.2390457218668787279944018975188698916
  190   0.23904572186687872799380568134648731574
  210   0.239045721866878727994638920381702081039
  180   0.2390457218668787279938038282759738764903
  240   0.23904572186687872799376343641233151780425
  200   0.239045721866878727993787227313448949620701
  210   0.2390457218668787279937643905514034511916807
  240   0.23904572186687872799376343647866135297401781
  270   0.239045721866878727993763435938861976903134646
  210   0.2390457218668787279937634359425941137423340287
  200   0.23904572186687872799376343820296064102403255952
  210   0.239045721866878727993763435984712680208947168584
  250   0.2390457218668787279937634826970431260165256813435
  260   0.23904572186687872799376343593884238423975204056029
$\endgroup$
11
  • $\begingroup$ Can't reproduce it in 13.0.0 on Windows 10: "NMinimize::nnum: The function value [Sqrt](Abs[<<1>>]^2+2 Abs[<<59>>+<<1>>-0.08610469561565808894 <<1>>]^2+2 Abs[4.6735580887155560026+<<44>> tesx-<<43>> Power[<<2>>]]^2) is not a number at {a0,a1,a2} = {0.6735580887155560026,0.6594922363525664855,0.08610469561565808894}." $\endgroup$
    – user64494
    Feb 9, 2022 at 4:29
  • $\begingroup$ @user64494 Well actually it should be Norm[tesy - a2 tesx^2 - a1 tesx - a0] to be the $L_2$ norm. However, that makes no difference. The Norm function is a vector treatment, and you would have to use that explicitly to obtain results. $\endgroup$
    – Carl
    Feb 9, 2022 at 4:38
  • $\begingroup$ If you replace MaxIterations -> 20000, by MaxIterations -> 137, the you will see the iterations ended at 137. I don't see much content in your question. $\endgroup$
    – user64494
    Feb 9, 2022 at 4:51
  • 3
    $\begingroup$ It's in code of Optimization`NMinimizeDump`CoreNM that convergence is checked every ten iterations. I suppose it's an expensive operation, so it is not done every step. If "PostProcess" is true, it may take a few more steps after the initial convergence. One may observe diagnostic Print statements if you set Optimization`NMinimizeDump`$DiagnosticLevel high enough, for instance Block[{Optimization`NMinimizeDump`$DiagnosticLevel = 3}, NMinimize[...]], you will see itr, the current number of iteration reported sometimes. (There's a lot of output, beware.) $\endgroup$
    – Michael E2
    Feb 9, 2022 at 5:20
  • 2
    $\begingroup$ @user64494 One has to be careful not to ask too many questions at once. The question may not be important for you, but I am not you, and I didn't know how to answer it by myself. I have enough of an answer here to know what to do, and I can do the rest by myself. The answer is useful to me, and I am satisfied with that response. $\endgroup$
    – Carl
    Feb 9, 2022 at 6:59

1 Answer 1

8
$\begingroup$

It's in code of Optimization`NMinimizeDump`CoreNM that convergence is checked every ten iterations. I suppose it's an expensive operation, so it is not done every step. The code may be inspected with GeneralUtilities`PrintDefinitions[Optimization`NMinimizeDump`CoreNM].

If "PostProcess" is true, it may take a few more steps after the initial convergence. One may observe diagnostic Print statements if you set Optimization`NMinimizeDump`$DiagnosticLevel high enough. For instance:

Block[{Optimization`NMinimizeDump`$DiagnosticLevel = 3}, 
 NMinimize[...]
 ]

You will see itr, the current number of iterations, reported sometimes. There's a lot of output, beware. The higher the setting the more output.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.