2
$\begingroup$

I have a function of several variables for which I'd like to find the maximum.

On one hand, it's easy to work with in the sense that it's smooth, and its only local maximum is a global one. On the other hand, it starts behaving weirdly if you stray too far from this maximum.

When I try running FindMaximum on it, it fails to give meaningful results unless I use an initial point which is very close to the maximum. I suspect that it tries to look at points which are too far and runs into trouble. I've tried several of the Methods described in the documentation.

If it just used a simple gradient ascent with small steps, it could easily find the maximum.

So my question is - how do I instruct FindMaximum to be more conservative in its search?

Edit: My actual function is complicated, but I can reproduce the effect with a much simpler one that captures the basic problem.

FindMaximum[(1 + a) (1 - a^2)^(1/10), {{a, 1/100}}]

The maximum of this function is at 5/6 (with a value of 1.62836), as can be easily seen by starting anywhere the function is defined and following the slope. But when I run the above command, I get the result {1.46441, {a -> 0.508985}} with the message:

"FindMaximum::nrnum: "The function value -3.46289-1.12516\ I is not a real number at {a} = {2.1873596654572154`}"

For some reason FindMaximum decided that trying to evaluate the function at 2.187, where it's undefined/complex-valued, is a good idea - and then went haywire.

What parameters can be set for FindMaximum to allow it to handle this function with this starting point? I suspect that whatever works here, might also work for my function.

|improve this question
$\endgroup$
  • 4
    $\begingroup$ This is likely function dependent, which means we won't be able to help without specifics. $\endgroup$ – march Jun 18 '16 at 19:39
  • $\begingroup$ @march I don't understand why it should be function dependent. I just want to run gradient descent and limit the step size taken. The specific step size will depend on the function, but that should be just a parameter that I can set. If there's no way to do it (without implementing the whole thing myself) that's a valid answer, though a disappointing one. $\endgroup$ – Meni Rosenfeld Jun 18 '16 at 19:45
  • $\begingroup$ It sounds like you know more about this than I do, which means I should suggest reading the Mathematica tutorials/documentation on NumericalNonlinearGlobalOptimization. $\endgroup$ – march Jun 18 '16 at 19:49
  • $\begingroup$ I've added an example function that, while much simpler than my actual one, exhibits similar behavior. $\endgroup$ – Meni Rosenfeld Jun 19 '16 at 9:46
3
$\begingroup$

Your question seems to assume a line search-type method. From the documentation, then:

Method -> {"QuasiNewton",
 "StepControl" -> {"LineSearch", "MaxRelativeStepSize" -> s}
}

Where s is some positive value.

If BFGS is too sophisticated for you, you could try using the conjugate gradient method instead. Indeed, there are a large number of possibilities for both the overall and the line search method, and of course for their combinations.

I just note that your supposition regarding the problem may or may not be correct. You should check that before you proceed, otherwise you will waste a lot of time trying different combinations of options.

|improve this answer
$\endgroup$
  • $\begingroup$ My supposition about the problem is correct, since by now I have already implemented my own very rudamentary search method, and it easily reached the solution starting from the very same starting point which FindMaximum failed on. Anyway, the MaxRelativeStepSize does seem relevant - if I limit it, FindMaximum does make some progress, instead of quitting at the first iteration - but it just stops part of the way through and doesn't get to the optimum, even if I increase MaxIterations, WorkingPrecision, AccuracyGoal and PrecisionGoal. $\endgroup$ – Meni Rosenfeld Jun 19 '16 at 9:32
  • $\begingroup$ @MeniRosenfeld your choice of options to change seems odd. Increasing AccuracyGoal and PrecisionGoal would delay convergence, rather than hastening it. WorkingPrecision is irrelevant. Only MaxIterations should make a difference. This answer still applies to your updated question, with s equal to 0.185 or less. I believe you about your problem, but my post directly answers your question; if your question is actually something different, then you should articulate that more clearly. What do you even mean by "just stops"? Converges prematurely? Try increasing "CurvatureFactor", etc. $\endgroup$ – Oleksandr R. Jun 19 '16 at 13:57
  • $\begingroup$ 1. I understand that MaxIterations is the most relevant one, but when it didn't help I tried others. I figured maybe it stopped because it was satisfied with the solution, and that demanding higher precision might prod it to continue towards a better one. 2. By "just stops" I mean that it doesn't display any error messages, but returns a result which is clearly far from the optimum. 3. It's possible my example function was too easy, I'll try coming up with a harder one that demonstrates the problem more clearly. 4. I'll try the CurvatureFactor thing. $\endgroup$ – Meni Rosenfeld Jun 19 '16 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.