1
$\begingroup$

I'm attempting to calculate the gradient of a function defined by two variables. After following a tutorial I found, I have the following code:

f[x_, y_] = (2 x^4 - x^2 + 3 x - 7 y + 1)/(e^(2 x^2/3 + 3 y^2/4));
(*t is gradient of f*)
t[{x_, y_}] := {
D[f[x, y], x],
D[f[x, y], y]
};

(*
Trying to use NestList to basically calculate gradient descent 'n' times/
use Euler's method starting at the point (.510, -.445)
*)

NestList[t, {.51, -.445}, n]

However, the output I'm getting is ".51" and "-.445" are not valid variables. I understand this is because I when I apply NestList, I am essentially taking the derivative with respect to ".51" and "-.445" which isn't possible obviously. However, I am unsure of how to apply NestList with respect to the variables x and y at the point (.51,-.445).

$\endgroup$
  • $\begingroup$ try ClearAll[t];t[{x_, y_}] = {D[f[x, y], x], D[f[x, y], y]};n = 3; NestList[t, {.51, -.445}, n]? $\endgroup$ – kglr Nov 16 '19 at 3:27
  • $\begingroup$ .. or ClearAll[t2];t2[{x_, y_}] = Grad[f[x, y], {x, y}];n = 3;NestList[t2, {.51, -.445}, n]? $\endgroup$ – kglr Nov 16 '19 at 3:28
  • $\begingroup$ .. or ClearAll[t3];t3[{x_, y_}] := Evaluate[{D[f[x, y], x], D[f[x, y], y]}]; n = 3;NestList[t3, {.51, -.445}, n] $\endgroup$ – kglr Nov 16 '19 at 3:29
  • $\begingroup$ @kglr Yes, thank you that does work! I guess the "clear" did it... $\endgroup$ – H. Khan Nov 16 '19 at 4:09
3
$\begingroup$

What you have is not quite the way to do steepest descent. Recall that you need to pick a step size for the direction where the next point is headed. In fancy code, that is done through some form of line search, but to keep things simple here, let us use a fixed step size.

γ = 1/10; (* step size *)
iter[{x_, y_}] = {x, y} - γ D[f[x, y], {{x, y}}];

(Note that I have used here the special syntax of D[] for evaluating the gradient; of course, D[f[x, y], {{x, y}}] == {D[f[x, y], x], D[f[x, y], y]}.)

Generate the iterates until convergence and look at the last two:

its = FixedPointList[iter, {-0.8, 0.4}];
Take[its, -2]
   {{-0.414549, 0.791434}, {-0.414549, 0.791434}}

(If you only want a certain number of iterates, replace FixedPointList[] with NestList[].)

Visualize the steepest-descent path:

ContourPlot[f[x, y], {x, -1, 0}, {y, 0, 1}, 
            Epilog -> {Directive[Orange, Arrowheads[Small]], Arrow[its]}, 
            PlotRange -> All]

steepest descent

| improve this answer | |
$\endgroup$
1
$\begingroup$

Via @kglr:

ClearAll[t];
t[{x_, y_}] = {
D[f[x, y], x], 
D[f[x, y], y]
};
NestList[t, {.51, -.445}, n]
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.