I am comparing a result of optimization using R and Mathematica. Mathematica (NMaximize) has found a significantly better (global optimum) result then R (using L-BFGS-B).

The function has 18 variables. The R routine stopped at a place, where none of the gradient is zero. So it's not even a local optimum. I am trying to find out why.

I wonder if I can duplicate the same result in Mathematica using the same L-BFGS-B method?

  • 1
    $\begingroup$ Quasi-Newton methods use BFGS. reference.wolfram.com/mathematica/tutorial/… $\endgroup$ – Dr. belisarius Dec 7 '14 at 16:16
  • 4
    $\begingroup$ NMinimize uses direct search metaheuristics, not gradient-based methods, and specifically aims to find a global optimum rather than a local one (except perhaps in the case of the Nelder-Mead method). I don't know why R stopped, although maybe the history length was not long enough (there is no reason to use L-BFGS rather than full BFGS for a function of only 18 variables). Anyway, the comparison should be with FindMinimum, not NMinimize. $\endgroup$ – Oleksandr R. Dec 7 '14 at 20:10
  • $\begingroup$ @OleksandrR. The original paper was written in 2003. The goal was still to find a global optima. I will try FindMinimum on the same problem and see what happens. I think it may be it was just an old version of R? which caused it to stop at a bad place (or even a local). $\endgroup$ – Chen Stats Yu Dec 7 '14 at 22:57
  • 5
    $\begingroup$ Differential evolution (the most powerful method offered by NMinimize) was invented in 1995, while direct search methods started to be considered respectable for global optimization some time in the early 80s. (The Nelder-Mead method was invented in 1965.) So, really, a paper written in 2003 should not have used BFGS to try to find a global optimum, unless it was also known that the function is uniformly convex. But, if so, (L-)BFGS should not stop. So, it sounds like either a bug or a bad choice of method. $\endgroup$ – Oleksandr R. Dec 8 '14 at 1:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.