10
$\begingroup$

Let's say I want to calculate the following scalar-by-matrix derivative

$$\frac{\partial}{\partial A} \text{tr} \left[(\vec X^T A)^T (\vec X^T A)\right],$$

with $\vec X$ and $A$ being a $n \times 1$ and a $n \times m$ matrix, respectively. Is there a way in Mathematica to get the result

$$2 \vec X (\vec X^T A)$$

without explicitly defining (for instance)

n=3
m=2
A=Array[a,{n,m}]
X=Array[x,{n,1}]

and calculating

D[Tr[Transpose[Transpose[X].A].Transpose[X].A],{A}]

? The problem with this approach is that the Mathematica result cannot be easily cast back into an human-readable form like

$$2 \vec X (\vec X^T A).$$

$\endgroup$
6
$\begingroup$

Your expression simplifies to this $$\vec X \vec X^T A + A^T \vec X \vec X^T$$ using just these rules

Unprotect[D, Transpose, Dot];
(*Derivative rules*)
D[Tr[A_], X_] := Tr[D[A, X]]
D[Transpose[A_], X_] := D[A, X]\[Transpose]
D[A_ .B_, X_] := D[A, X].B + A.D[B, X]

(*Tranpose rules*)
0\[Transpose] := 0
1\[Transpose] := 1
(A_\[Transpose])\[Transpose] := A
(A_ .B_)\[Transpose] := B\[Transpose].A\[Transpose]

(*Dot rules*)
Dot[_, 0] := 0
Dot[0, _] := 0
Dot[1, A_] := A
Dot[A_, 1] := A
Protect[D, Transpose, Dot];

If it's correct then I'm quite sure these two parts are different (transpose of one another), so you can't add them to get coefficient 2.

$\endgroup$
8
  • 1
    $\begingroup$ Thank you very much for your answer. Are you sure that the first rule is correct? I think that the matrix derivative of the trace (i.e. a scalar) should give a matrix, whereas the trace of the tensor D[A,X] will give a scalar. You would have different objects on both sides of D[Tr[A_], X_] := Tr[D[A, X]]. $\endgroup$
    – cryo111
    Apr 11 '14 at 12:55
  • 1
    $\begingroup$ @cryo111 Wouldn't a matrix derivative of a matrix be a fourth rank tensor? Therefore its trace should be a matrix, not a scalar. $\endgroup$
    – swish
    Apr 11 '14 at 18:06
  • 1
    $\begingroup$ The Mathematica help says that "For a higher-rank tensor, Tr gives the sum of elements with equal indices" (i.e., a scalar). $\endgroup$
    – cryo111
    Apr 11 '14 at 19:25
  • 1
    $\begingroup$ But yes, a matrix-by-matrix derivative should give a fourth-rank tensor. $\endgroup$
    – cryo111
    Apr 11 '14 at 19:31
  • 1
    $\begingroup$ Until there is no actual tensor inside Tr it's just a symbolic expression, is up to you how to interpret it. $\endgroup$
    – swish
    Apr 11 '14 at 20:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.