# Can I use symbolic tensors for simple linear algebra and calculus?

As an example, I'd like to calculate this symbolic derivative:

$\frac{\partial}{\partial b}(x-A.b)^{\mathsf{T}}.(x-A.b)$

where $x$ and $b$ are vectors and $A$ is a matrix.

What I tried is this:

Clear[x, a, b, n, m]
\$Assumptions = {x ∈ Vectors[n, Reals], a ∈ Matrices[{n, m}], b ∈ Vectors[m, Reals]}

expr = TensorReduce[(x - a.b).(x - a.b)]


The value returned by TensorReduce looks ok, in the sense that TensorRank and TensorDimensions return 0 and {} as expected for a scalar, and when I use replacement rules to replace x, a, b with proper-sized arrays, I get the expected result.

However, when I write:

D[expr, b]


I get the expression: -2 x.a.1 + a.1.a.b + a.b.a.1. Looking at this expression, I can sort of guess what the derivative should look like, but I can't use it in code like that. For example, I can't use TensorDimensions to get the derivative's dimension, and I can't use replacement rules to insert arrays in place of the symbolic tensors.

I've also tried using TensorProduct/TensorContract instead of Dot:

xmab = TensorReduce[x - TensorContract[TensorProduct[a, b], {{2, 3}}]];
expr = TensorReduce[
TensorContract[TensorProduct[xmab, xmab], {{1, 2}}]];


This gives a sensible looking expr, but I think D doesn't work with TensorProduct/TensorContract:

D[expr, {b, 1}]


-2 (TensorContract^(1,{{0,0},{0,0}}))[a[TensorProduct]b[TensorProduct]x,{{1,4},{2,3}}] (TensorProduct^(0,1,0))[a,b,x]+(TensorContract^(1,{{0,0},{0,0},{0,0}}))[a[TensorProduct]a[TensorProduct]b[TensorProduct]b,{{1,3},{2,6},{4,5}}] ((TensorProduct^(0,0,0,1))[a,a,b,b]+(TensorProduct^(0,0,1,0))[a,a,b,b])

• I think calculus functions (D) don't work with tensors. D seems to work with Dot, but it does give a weird looking result that I don't understand: 1 . x (anyone copying this, note the space around the .). This type of expression will never evaluate 1 . 2 or 1 . {1,2}. I would have expected Dot to act as Times with scalars ... – Szabolcs Jul 7 '14 at 15:02
• @Szabolcs: I've noticed that Dt[f.g, x] is f.Dt[g, x] + Dt[f, x].g: MMA seems to apply the product rule to Dot, so I'd say the 1 in 1 . x should be the Jacobian D[x,x], i.e. IdentityMatrix. (Did that make sense?). So I could probably safely replace Dot[x_,1] -> x. But I can't get the transpositions right, because D doesn't seem to work with Transpose, Transpose doesn't distribute over sums and so on. – Niki Estner Jul 7 '14 at 15:40