I would like to compute gradients of tensor expressions without having to instantiate particular tensors and for arbitrary sizes.

Example 1 : For example, let's say I want to minimize $C=\sum_j(\sum_i M_{ij}X_i-Y_i)^2$ where $M$ is a matrix and $X$ and $Y$ are vectors, I would compute the gradient $$ \frac{\partial C}{\partial X_k}=2\sum_jM_{kj}\left(\sum_i M_{ij}X_i-Y_j\right).$$

Of course I can compute this trivial one by hand. But what if I had a much more complicated expression with tensors of higher dimension?

Example 2 :Consider the covariance matrix: $$q_{jk}=\frac{1}{N-1}\sum_i \left(x_{ij}-\frac{1}{N}\sum_l x_{lj}\right)\left(x_{ik}-\frac{1}{N}\sum_l x_{lk}\right)$$ How can I compute its gradient with respect to the data $\frac{\partial q_{ik}}{\partial x_{mn}}$ ? This is a rank 4 tensor so I would like the result to be fully symbolic, i.e not for a particular range of indices but with symbolic indices.

How can I use Mathematica to compute such expressions ?

  • 1
    $\begingroup$ Perhaps an example of such a more complicated case could help. $\endgroup$
    – MarcoB
    Commented Mar 12 at 13:46
  • $\begingroup$ @MarcoB I have edited the question with a more complicated example $\endgroup$
    – Nichola
    Commented Mar 13 at 16:43

1 Answer 1


You may do this elementwise like e.g.:

n = 3;
xs = Array[x, n];
ys = Array[y, n];
ms = Array[m, {n, n}];
c = Total[(xs . ms - ys)^2];
D[c, {ys}] // MatrixForm

enter image description here

  • $\begingroup$ Is there no way to do this in some more general notation without having to specify n=.. ? $\endgroup$
    – Nichola
    Commented Mar 12 at 12:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.