# Hierarchical optimization

I am looking to find a point of a function $f(x,y)$ that minimizes $f$ with respect to $x$ and at the same time fulfills $x=y$ (but the point is not necessarily a minimum with respect to $y$). I would consider this a hierarchical optimization. Here a simple example:

FindRoot[y == x /. NMinimize[(x - y^2)^2, x][], {y, 1}]


This code does not work. What I intend to do is find $y$ such that $y==x$ and $x$ minimizes $f(x,y)=(x-y^2)^2$. The solution would be $y=1$. Is there a syntactically correct way to do this in Mathematica?

Edit: The example was poorly chosen because $f$ has a minimum with respect to $y$ as well at $x=y=1$. Here is a modified example:

f[x_, y_] := (x - y^2)^2 + Exp[y]


This is the result I would like to obtain:

Block[{y = 1}, NMinimize[f[x, y], x][]]
(* {x -> 1.} *)


However, minimizing $f[x,x]$ leads to a wrong solution:

NMinimize[{f[x, x], x > 0.5}, x]
(* {x -> 0.5} *)


With pen-and-paper math, the solution I am aiming for reads

1. Minimize with $f$ respect to $x$ yields $x=y^2$.

2. Solve $x=y$ gives $y=y^2$ or $y=0,1$.

I am looking for a numerical method to obtain the same result.

• Only use NMinimize and replace y with x – Jack LaVigne Oct 21 '16 at 14:51
• For my first example this approach worked. I have modified the example to better illustrate my problem. – Felix Oct 21 '16 at 15:41

f[x_, y_] := (x - y^2)^2 + Exp[y]
g[y_?NumericQ] := x /. NMinimize[f[x, y], x][]
FindRoot[g[y] == y , {y, 1.25}]


{y->1.}

• Yes, this is exactly what I was looking for. Why does it not work when writing it in one line, i.e., without the auxiliary function g? – Felix Oct 21 '16 at 18:41
• the g function ensures that NMinimize is only evaluated with a numeric y value. – george2079 Oct 21 '16 at 18:46

You can find the minimum by taking the derivative and setting equal to zero.

f[x_, y_] := (x - y^2)^2 + Exp[y];
sol = Solve[D[f[x, y], x] == 0, x]


This gives {{x -> y^2}}, and you now want to solve for when both sides of this are equal:

Solve[y == First@(x /. sol), y]

{{y -> 0}, {y -> 1}}

• Unfortunately, this analytical approach does not work for my actual problem, because Solve is limited to very simple (polynominal) functions. I need a numerical method. – Felix Oct 21 '16 at 18:20

Starting with Bill S's answer, you can combine the steps using Solve

Solve[{D[f[x, y], x] == 0, x == y}, {x, y}]
(* {{x -> 0, y -> 0}, {x -> 1, y -> 1}} *)


or use Reduce which tends to be a bit more robust

Reduce[2 (x - y^2) == 0 && x == y, {x, y}]
(* (x == 0 || x == 1) && y == x *)


and if you need a numerical solution

NSolve[{D[f[x, y], x] == 0, x == y}, {x, y}, Reals]
(* {{x -> 1., y -> 1.}, {x -> 0., y -> 0.}} *)

• +1 for a numerical solution that is also much faster than george's version. George's solution, however, works with more complicated functions, where NMinimize is required to find the minimum (solving for derivative == 0 often fails when the functions involved are non-trivial). – Felix Oct 22 '16 at 9:56