1
$\begingroup$

I'm attempting to calculate the gradient of a function defined by two variables. After following a tutorial I found, I have the following code:

f[x_, y_] = (2 x^4 - x^2 + 3 x - 7 y + 1)/(e^(2 x^2/3 + 3 y^2/4));
(*t is gradient of f*)
t[{x_, y_}] := {
D[f[x, y], x],
D[f[x, y], y]
};

(*
Trying to use NestList to basically calculate gradient descent 'n' times/
use Euler's method starting at the point (.510, -.445)
*)

NestList[t, {.51, -.445}, n]

However, the output I'm getting is ".51" and "-.445" are not valid variables. I understand this is because I when I apply NestList, I am essentially taking the derivative with respect to ".51" and "-.445" which isn't possible obviously. However, I am unsure of how to apply NestList with respect to the variables x and y at the point (.51,-.445).

$\endgroup$
4
  • $\begingroup$ try ClearAll[t];t[{x_, y_}] = {D[f[x, y], x], D[f[x, y], y]};n = 3; NestList[t, {.51, -.445}, n]? $\endgroup$
    – kglr
    Nov 16, 2019 at 3:27
  • $\begingroup$ .. or ClearAll[t2];t2[{x_, y_}] = Grad[f[x, y], {x, y}];n = 3;NestList[t2, {.51, -.445}, n]? $\endgroup$
    – kglr
    Nov 16, 2019 at 3:28
  • $\begingroup$ .. or ClearAll[t3];t3[{x_, y_}] := Evaluate[{D[f[x, y], x], D[f[x, y], y]}]; n = 3;NestList[t3, {.51, -.445}, n] $\endgroup$
    – kglr
    Nov 16, 2019 at 3:29
  • $\begingroup$ @kglr Yes, thank you that does work! I guess the "clear" did it... $\endgroup$
    – H. Khan
    Nov 16, 2019 at 4:09

2 Answers 2

4
$\begingroup$

What you have is not quite the way to do steepest descent. Recall that you need to pick a step size for the direction where the next point is headed. In fancy code, that is done through some form of line search, but to keep things simple here, let us use a fixed step size.

γ = 1/10; (* step size *)
iter[{x_, y_}] = {x, y} - γ D[f[x, y], {{x, y}}];

(Note that I have used here the special syntax of D[] for evaluating the gradient; of course, D[f[x, y], {{x, y}}] == {D[f[x, y], x], D[f[x, y], y]}.)

Generate the iterates until convergence and look at the last two:

its = FixedPointList[iter, {-0.8, 0.4}];
Take[its, -2]
   {{-0.414549, 0.791434}, {-0.414549, 0.791434}}

(If you only want a certain number of iterates, replace FixedPointList[] with NestList[].)

Visualize the steepest-descent path:

ContourPlot[f[x, y], {x, -1, 0}, {y, 0, 1}, 
            Epilog -> {Directive[Orange, Arrowheads[Small]], Arrow[its]}, 
            PlotRange -> All]

steepest descent

$\endgroup$
1
$\begingroup$

Via @kglr:

ClearAll[t];
t[{x_, y_}] = {
D[f[x, y], x], 
D[f[x, y], y]
};
NestList[t, {.51, -.445}, n]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.