# Gradient Descent's Update Rule for Multiparameter Functions

I'm sure this has been answered before, but I haven't been able to find it. I'm trying to manually implement gradient descent for some function, f, with a vector of any length.

I define the update rule as follows for a vector with two entries:

a = {0.1, 0.01};
Step[th_] := th + a Grad[f[{x, y}], {x, y}] /. {x -> th[[1]], y ->th[[2]]}

My question then becomes, how would I do this for a vector of n-dimensions without having to manually key in all the replacements? Ideally, I would just check the dimension of my input parameter, and then follow through with the correct calculations. How can I achieve that?

• Ponder on the following snippet: With[{vec = {u, v, w}}, vec + (a (Derivative[##][f] @@ vec) & @@@ IdentityMatrix[Length[vec]])] – J. M. is away Jun 15 '16 at 12:36
• Thanks @J.M. It didn't evaluate completely though. I had to change it to the following: Step[th] := With[{x = th}, x + a ((Derivative[##][l] @@ {x}) & /@ IdentityMatrix[Length[x]])]. For some reason, the apply function didn't evaluate my function, and I had to change it to the map. – Frederik Brinck Jensen Jun 15 '16 at 13:52
• Yes, your modifications are the right one for the case where the argument is a $k$-vector instead of a function with $k$ arguments. – J. M. is away Jun 15 '16 at 16:15