# Symbolic scalar-by-matrix derivative

Let's say I want to calculate the following scalar-by-matrix derivative

$$\frac{\partial}{\partial A} \text{tr} \left[(\vec X^T A)^T (\vec X^T A)\right],$$

with $\vec X$ and $A$ being a $n \times 1$ and a $n \times m$ matrix, respectively. Is there a way in Mathematica to get the result

$$2 \vec X (\vec X^T A)$$

without explicitly defining (for instance)

n=3
m=2
A=Array[a,{n,m}]
X=Array[x,{n,1}]


and calculating

D[Tr[Transpose[Transpose[X].A].Transpose[X].A],{A}]


? The problem with this approach is that the Mathematica result cannot be easily cast back into an human-readable form like

$$2 \vec X (\vec X^T A).$$

-

Your expression simplifies to this $$\vec X \vec X^T A + A^T \vec X \vec X^T$$ using just these rules

Unprotect[D, Transpose, Dot];
(*Derivative rules*)
D[Tr[A_], X_] := Tr[D[A, X]]
D[Transpose[A_], X_] := D[A, X]\[Transpose]
D[A_ .B_, X_] := D[A, X].B + A.D[B, X]

(*Tranpose rules*)
0\[Transpose] := 0
1\[Transpose] := 1
(A_\[Transpose])\[Transpose] := A
(A_ .B_)\[Transpose] := B\[Transpose].A\[Transpose]

(*Dot rules*)
Dot[_, 0] := 0
Dot[0, _] := 0
Dot[1, A_] := A
Dot[A_, 1] := A
Protect[D, Transpose, Dot];


If it's correct then I'm quite sure these two parts are different (transpose of one another), so you can't add them to get coefficient 2.

-
Thank you very much for your answer. Are you sure that the first rule is correct? I think that the matrix derivative of the trace (i.e. a scalar) should give a matrix, whereas the trace of the tensor D[A,X] will give a scalar. You would have different objects on both sides of D[Tr[A_], X_] := Tr[D[A, X]]. –  cryo111 Apr 11 '14 at 12:55
@cryo111 Wouldn't a matrix derivative of a matrix be a fourth rank tensor? Therefore its trace should be a matrix, not a scalar. –  swish Apr 11 '14 at 18:06
The Mathematica help says that "For a higher-rank tensor, Tr gives the sum of elements with equal indices" (i.e., a scalar). –  cryo111 Apr 11 '14 at 19:25
But yes, a matrix-by-matrix derivative should give a fourth-rank tensor. –  cryo111 Apr 11 '14 at 19:31
Until there is no actual tensor inside Tr it's just a symbolic expression, is up to you how to interpret it. –  swish Apr 11 '14 at 20:27