I would like to compute gradients of tensor expressions without having to instantiate particular tensors and for arbitrary sizes.

Example 1 : For example, let's say I want to minimize $$C=\sum_j(\sum_i M_{ij}X_i-Y_i)^2$$ where $$M$$ is a matrix and $$X$$ and $$Y$$ are vectors, I would compute the gradient $$\frac{\partial C}{\partial X_k}=2\sum_jM_{kj}\left(\sum_i M_{ij}X_i-Y_j\right).$$

Of course I can compute this trivial one by hand. But what if I had a much more complicated expression with tensors of higher dimension?

Example 2 :Consider the covariance matrix: $$q_{jk}=\frac{1}{N-1}\sum_i \left(x_{ij}-\frac{1}{N}\sum_l x_{lj}\right)\left(x_{ik}-\frac{1}{N}\sum_l x_{lk}\right)$$ How can I compute its gradient with respect to the data $$\frac{\partial q_{ik}}{\partial x_{mn}}$$ ? This is a rank 4 tensor so I would like the result to be fully symbolic, i.e not for a particular range of indices but with symbolic indices.

How can I use Mathematica to compute such expressions ?

• Perhaps an example of such a more complicated case could help. Commented Mar 12 at 13:46
• @MarcoB I have edited the question with a more complicated example Commented Mar 13 at 16:43

You may do this elementwise like e.g.:

n = 3;
xs = Array[x, n];
ys = Array[y, n];
ms = Array[m, {n, n}];
c = Total[(xs . ms - ys)^2];
D[c, {ys}] // MatrixForm


• Is there no way to do this in some more general notation without having to specify n=.. ? Commented Mar 12 at 12:35