Calculating Matrix Derivatives

Question

I'm sorry if this is an easy question, but I'm trying to refresh my memory on calculating derivatives of matrix products and have been reading the Matrix Cookbook. I'd like to rebuild my intuition by replicating the results in Section 2.4.1 on page 10 where the authors list some matrix derivative results. I found this link: Differentiating functions of vectors/matrices? which helped some, but I think I need to get back to basics.

If I have the function:

$f(x) = \dfrac{1}{2}x^{T} A x + b^{T} x $

I want to calculate $\nabla f(x)$ for generic matrices $x$, $b$ and $A$. After this I want to calculate the Hessian.

How can I do this in Mathematica?

Answer 1

Here is one way, assuming $A$ is symmetric:

(* construct vectors and matrix *)
n=2;
x=Array[xc,n];
b=Array[bc,n];
A=Array[Sort[Ac[##]]&,{n,n}];

(* define function and calculate derivatives *)
f=1/2*x.A.x+b.x;
grad=Grad[f,x]//Expand
hessian=Grad[grad,x]//Expand

(* check agreement with well-known formulas *)
grad==A.x+b
hessian==A

See also the documentation of Grad, under Examples/Applications.

Answer 2

Use the new-in-14.1 VectorSymbol and MatrixSymbol:

x = VectorSymbol["x", n];
b = VectorSymbol["b", n];
A = MatrixSymbol["A", {n, n}];
f = 1/2*x . A . x + b . x;

grad = Grad[f, x]

hessian = Grad[grad, x]

Answer 3

E.g.:

n = 3;
xs = Array[Subscript[x, #] &, n];
bs = Array[Subscript[b, #] &, {n}];
as = Array[Subscript[a, ##] &, {n, n}];
f[x_] = 1/2 x . as . x + bs . x;

grad = Grad[f[xs], xs]

hes = D[f[xs], {xs, 2}]