# Strategies to avoid LessEqual::nord in NMinimize?

When using NMinimize on functions with complex intermediate expressions (but a real end result), quite often one gets the error LessEqual::nord. Example:

NMinimize[Abs[(a+I b)^(3/2)],{a,b}]
(*
LessEqual::nord: Invalid comparison with 0.306069 + 0. I attempted.

LessEqual::nord: Invalid comparison with 0.306069 + 0. I attempted.

LessEqual::nord: Invalid comparison with 0.306069 + 0. I attempted.

General::stop: Further output of LessEqual::nord
will be suppressed during this calculation.

Less::nord: Invalid comparison with -0.0745302 + 0. I attempted.

NMinimize::cvmit:
Failed to converge to the requested accuracy or precision within 100
iterations.

{1.0635969220476164*^-12 + 0.*I, {a -> -8.71759358950322*^-9, b -> 5.707170335837908*^-9}}
*)


In some cases (not the one above; I didn't find a simple one where it happens) this also results in a clearly wrong result. Therefore it's desirable to remove the error messages.

Now the only way to get rid of this error is to change the expression in a way that it doesn't trigger the error. However the expressions are generally complicated enough that it's not feasible by hand. I've found that a combination of the following strategies works sometimes:

• Use ComplexExpand with the option TargetFunctions->{Re,Im}.
• Put the entire expression into an Abs or Re (despite it being known to be real from construction) and use Simplify or FullSimplify with appropriate constraints (and hope it finishes in reasonable time). (Abs of course only works if the result is also nonnegative)

However those strategies are not always sufficient. Therefore my question:

What are other good strategies to get the expression into a form suitable to NMinimize?

-

I think you have to do the same as in many such cases: protect your arguments to be strictly numerical:

f[a_?NumericQ, b_?NumericQ] := Abs[(a + I b)^(3/2)];


And then no problems:

NMinimize[f[a,b],{a,b}]

(*
==>  {1.11868*10^-26,{a->3.9489*10^-18,b->3.07007*10^-18}}
*)


Edit:

The following function automatically packs the expression into a function with _?NumericQ pattern arguments:

NOptimize[optfunc_,expr_,vars_,options___]:=
Module[{f,
varlist=If[ListQ[vars],vars,{vars}],
expression=If[ListQ[expr],First@expr,expr],
conditions=If[ListQ[expr],Rest@expr,{}]},
Evaluate[f@@(Pattern[#,_?NumericQ]&/@varlist)]=expression;
optfunc[{f@@varlist}~Join~conditions,vars,options]]


It can be used as follows:

NOptimize[NMinimize, a^2, a, AccuracyGoal->0.01]
(*
--> {2.39829*10^-33,{a->4.89724*10^-17}}
*)


or with constraints:

NOptimize[NMinimize, {a^2, a>3}, a, AccuracyGoal->0.01]
(*
--> {9.,{a->3.}}
*)


The following shows that it indeed solves the problem with LessEqual::Nord:

NOptimize[NMinimize,Abs[(a+I b)^(3/2)],{a,b}]
(*
--> {9.06219*10^-27,{a->4.31982*10^-18,b->4.8223*10^-19}}
*)

-
_?NumericQ belongs at the top of the FAQ. –  Mr.Wizard Jan 22 '12 at 19:24
@Mr.Wizard Agree. –  Leonid Shifrin Jan 22 '12 at 19:25
@Mr.Wizard That's what I said here: mathematica.stackexchange.com/questions/333/… . –  Szabolcs Jan 22 '12 at 19:37
@Leonid But to be honest, I really don't understand what's going on here and how symbolic evaluation can give this result. Perhaps NMinimize symbolically pre-processes the argument. –  Szabolcs Jan 22 '12 at 19:38
Astonishing. I would never have guessed that it is caused by passing non-numerical values. From the error, I thought that the + 0.*I was the problem. But with _?NumericQ it indeed works! –  celtschk Jan 22 '12 at 19:45

(This is a bit too long for a comment.)

Compare with:

FindMinimum[Abs[(a + I b)^(3/2)], {{a, 1}, {b, 1}}]
{1.4576121587694715*^-12, {a -> -9.089664269711015*^-9, b -> -9.09099600216378*^-9}}


but as you have seen, the version with NMinimize[] doesn't work. A look at the functions' Attributes[] sheds some light:

Attributes[NMinimize]

Attributes[FindMinimum]
{HoldAll, Protected}


That HoldAll in FindMinimum[] spells the difference. The HoldAll attribute allows for nonstandard evaluation of the objective function within the internals of FindMinimum[]. NMinimize[], on the other hand, sees the complex numbers within the objective function. Since part of what NMinimize[] does before the optimization is to start with the initial region $-1 \leq a,b \leq 1$ (which it does in the absence of explicit constraints), there is inevitably the interaction of complex numbers with comparison operators, leading to the observed warning messages.

-
The more experienced users will understand the implications of HoldAll, but the less experienced users will not. Could you expand the answer a bit? –  rcollyer Jan 23 '12 at 2:36
Sure.$\phantom{}$ –  Ｊ. Ｍ. Jan 23 '12 at 2:55
Good references, +1. –  rcollyer Jan 23 '12 at 2:58
The HoldAll attribute is the crux of the problem. See also Michael Trott's Numerics book, section 1.6: "No good excuse is available why NDSolve and NMinimize do not have the HoldAll attribute. Both functions have dummy variables that should be scoped from the surrounding." –  Andreas Lauschke Mar 16 '13 at 18:39

If I interpret your question as about getting rid of error messages ("it's desirable to remove the error messages"), you may use Off:

Off[LessEqual::nord];
Off[Less::nord];
NMinimize[Abs[(a + I b)^(3/2)], {a, b}]
(*
NMinimize::cvmit: Failed to converge to the requested accuracy or precision
within 100 iterations.

{1.0636*10^-12 + 0. I, {a -> -8.71759->10^-9, b -> 5.70717->10^-9}}

*)

-
Well, I also wrote that the reason why I want to get rid of them is that it gives sometimes wrong results. Which clearly shows that I wanted to actually get rid of the error, not just the messages, because suppressing the messages doesn't fix the result. –  celtschk Jan 22 '12 at 19:53
Well, you actually wrote "In some cases this also results in a clearly wrong result. Therefore...", which I interpreted as "the result is obviously wrong, so I don't need messages cluttering up my display, I know it!". Stupid me :) –  acl Jan 22 '12 at 19:58
I wasn't aware that my text could be misinterpreted that way. Of course the main concern is that I might also get wrong results without noticing it. –  celtschk Jan 22 '12 at 20:00
@celtschk no, it was probably stupidity on my part. However, you could try this: ClearAll[f];f[(a_)?NumericQ, (b_)?NumericQ] := (Sow[{a, b}]; Abs[(a + I*b)^(3/2)]); Select[Last[Last[Reap[NMinimize[f[a, b], {a, b}]]]], #1 ==Im[#1] & ] to see that it never actually does get a complex input. Or Trace it, but the output of that is too large. –  acl Jan 22 '12 at 20:11
FWIW I too read that piece of the question as wanting to suppress the error messages, not necessarily get rid of the errors themselves. So I don't think you should blame stupidity, acl, just misinterpretation. –  David Z Jan 22 '12 at 20:46