Symbolic linear algebra gradients/matrix calculus

Question

Can Mathematica generate symbolic expressions for gradients?

For example, if $x_1$ and $x_2$ are two points, could I get Mathematica to generate expressions similar to the following?

$\frac{\partial \left|x_1 - x_2\right|}{\partial x_1} = \frac{x_1 - x_2}{\left|x_1 - x_2\right|}$

$\frac{\partial \left|x_1 - x_2\right|}{\partial x_2} = \frac{x_2 - x_1}{\left|x_1 - x_2\right|}$

My experience with Mathematica is limited. I know how to get derivatives w.r.t. scalars.

Elsewhere on this site I found this question and from the answers it looks like Mathematica recently acquired the ability to do some amount of symbolic linear algebra.

Answer 1

Everything you want in the question can be done by defining the derivative of the Norm:

Derivative[1][Norm][z_] := z/Norm[z]

D[Norm[x - y], {x}]
(* ==> (x - y)/Norm[x - y] *)

Simplify[D[Norm[x - y], {y}]]
(* ==> (-x + y)/Norm[x - y] *)

Here, the syntax I used for the derivatives is such that it would remain valid if x or y were replaced by vectors (i.e., Lists).

Answer 2

To some extent (and with some care) this can be done with FeynCalc. At least I used it several times when I needed to compute gradients and divergences of Cartesian vectors. The trick is to work with D-dimensional 4-vectors and take the limit $D \to 3$ at the end. Since FeynCalc doesn't distinguish between upper and lower indices, the results are the same as if one would work with Cartesian vectors. For example, computing $\partial x_1^i/ \partial x_1^j$ via

FourDivergence[FVD[x1, i], FVD[x1, j]]

returns $g^{i j}$ which one should interpret as a Kronecker delta. For the examples from the OP's question we can use

FourDivergence[Sqrt[SPD[x1 - x2]], FVD[x1, i]] // Together

to get

$\frac{\text{x1}^i-\text{x2}^i}{\sqrt{\text{x1}^2-2 (\text{x1} \cdot \text{x2} \cdot )+\text{x2}^2}}$

and

FourDivergence[Sqrt[SPD[x1 - x2]], FVD[x2, i]] // Together

to obtain

$\frac{\text{x2}^i-\text{x1}^i}{\sqrt{\text{x1}^2-2 (\text{x1} \cdot \text{x2} \cdot )+\text{x2}^2}}$

This is of course not a general solution, but I find it quite useful from time to time, so why not share it here. If one wants something more general, then the xTensor package might be worth a try.