I want to find the following saddle point of the function of 10 variables:


I am not able to find the stationary points by setting the first derivatives to 0, so I need do optimize the function numerically. I think that that performing the above-mentioned optimization is equivalent to finding a set of {a1,a2,a3,a4,a5} that would maximize the following function:


That is how I would do the inner minimization, so I get the value of that function for the values of a1,a2,a3,a4 and a5 I provide as an input. Does anybody know how to find in practice a set of a1, a2, a3, a4 and a5 that would maximize the InnerMin function, or in the other words how to perform the outer optimization numerically (coupled with the inner minimization via the InnerMin function)?


1 Answer 1


What you have proposed works in principal:

inner[y_ /; NumericQ[y]] := (
                  lastx =  NMinimize[-7 - 6 x + x^2 - 8 y - y^2, x];
                  First@lastx )
NMaximize[inner[y] , y ]

{-1.77636*10^-15, {y -> -4.}} {-1.77636*10^-15, {x -> 3.}}}

When you see how slow this is ( a whole minute) with my simple example I think you'll reconsider though..

Re comment, here is a 4-variable example:

 inner[y_, z_] /; And @@ NumericQ /@ {y, z} := (
        lastwx = 
          -4 w + 6 w^2 - 4 w^3 + w^4 - 4 x + x^2 + 108 y 
               - 54 y^2 + 12 y^3 - y^4 + 8 z - z^2 , {w, x}];
           First@lastwx )
 NMaximize[inner[y, z], {y, z}]

{92., {y -> 2.99739, z -> 4.}}

{92., {w -> 1.00001, x -> 2.}}

Exact answer should be 1,2,3,4 ..

This only works because we know a a priori which variables to min/max and the function is really nice with just one stationary point.

I think for the general problem you should be doing your own gradient descent.

  • $\begingroup$ Thanks a lot, what should I put in my case for NumericQ[y], if I predifined function f[a1_,a2_,a3_,a4_,a5_,y1_,y2_,y3_,y4_,y5_], and want to optimize it how I have stated in the question. Just to make sure I don't make a mistake in the input, is this correct: inner[a1_, a2_, a3_, a4_, a5_ /; NumericQ[{a1,a2,a3,a4,a5}]] := (lastx = NMinimize[ f[a1, a2, a3, a4, a5, b1, b2, b3, b4, b5], {b1, b2, b3, b3, b4, b5}]; First@lastx) NMaximize[inner[a1, a2, a3, a4, a5], {a1, a2, a3, a4, a5}] lastx. It gives me some results, and reports the errors: $\endgroup$
    – ogledala
    Commented Feb 7, 2014 at 22:50
  • $\begingroup$ NMinimize::cvdiv: Failed to converge to a solution. The function may be unbounded. Are therefore results meaningful? $\endgroup$
    – ogledala
    Commented Feb 7, 2014 at 22:53
  • $\begingroup$ It is so strange to me, I have not changed anything and know the error is: NMaximize::nnum: The function value -inner[0.643476,0.280303,0.964763,0.766972] is not a number at {b1,b2,b3,b4} = {0.643476,0.766972,0.280303,0.964763}. >>. Any ideas how to tackle this? @george2079 $\endgroup$
    – ogledala
    Commented Feb 8, 2014 at 0:15
  • $\begingroup$ i'd suggest working on a simple solvable problem (eg w^2+x^2-y^2-z^2 ) to sort out the syntax. Your errors relate to your function which you havent given.. But really i think you need to do something more sophisticated. (study the links Daniel posted for starters ) $\endgroup$
    – george2079
    Commented Feb 8, 2014 at 13:12
  • $\begingroup$ Before going to something sophisticated, I'd like to understand the principles of your code. Here is what I did with your example: f[x_, y_, z_, w_] := w^2 + x^2 - y^2 - z^2 inner[x_, y_ /; NumericQ[{x, y}]] := (lastx = NMinimize[f[x, y, z, w], {z, w}]; First@lastx) NMaximize[inner[x, y], {x, y}] lastx and this was the error message: NMaximize::nnum: The function value -inner[0.918621,0.716689] is not a number at {x,y} = {0.918621,0.716689}. >> What I am doing wrong? $\endgroup$
    – ogledala
    Commented Feb 8, 2014 at 21:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.