I'm taking a machine learning course, which involves taking a lot of analytical gradients and Hessians. It would be ideal if I could perform these calculations in Mathematica. However, I am only aware of how to do this for scalar functions, not ones that involve matrices. Could anybody assist me with this or point me in the right direction?

For example, say I have

$$g(\boldsymbol{w})=-\cos(2\pi\boldsymbol{w}^{T}\boldsymbol{w})+\boldsymbol{w}^{T}\boldsymbol{w}$$

I want to have Mathematica be able to compute, symbolically, that the gradient is

$$\nabla g(\boldsymbol{w})=\sin(2\pi\boldsymbol{w}^{T}\boldsymbol{w})4\pi\boldsymbol{w}+2\boldsymbol{w}$$

and, ideally, that the Hessian is

$$\nabla^2g(\boldsymbol{w})=(4\pi)^{2}\cos(2\pi\boldsymbol{w}^{T}\boldsymbol{w})\boldsymbol{w}\boldsymbol{w}^{T}+(2+4\pi\sin(2\pi\boldsymbol{w}^{T}\boldsymbol{w}))\boldsymbol{I}$$

• Take a good look at D, you can use vector and tensor valued gradients. For example d = 3; w = Array[wc, {d}]; g = w.w; D[g, {w, 2}] – Mauricio Fernández Apr 16 '17 at 9:43
• @MauricioLobos What do you propose when the dimension of the vector is not known (say $n$)? – wolfies Apr 16 '17 at 10:41
• No idea, I dont know if Mathematica can evaluate vector expressions like that. – Mauricio Fernández Apr 16 '17 at 10:52
• Possibly related: some calculus: (3242), (19596), (19615), (135489); just vectors: (73990), (117941) – Michael E2 Apr 16 '17 at 12:25
• Specifically for gradients and hessians have a look at mathematica.stackexchange.com/a/16378/1089 – chris Apr 16 '17 at 15:13