I am doing computations with multilinear maps and representing them with tensors. For example, I have an operator $O : V_1 \otimes V_2 \to W$, where $\dim(V_1) = \dim(V_2) = 5$ and $\dim(W)=3$ that I represent by a tensor of rank $(3,5,5)$:
op = Array[o, {3, 5, 5}]
I want to compose it with two operators $F_1 : U_1 \to V_1$ and $F_2 : U_2 \to V_2$, where e.g., $\dim(U_1) = 10$ and $\dim(U_2) = 8$ to get $O \circ (F_1 \otimes F_2) : U_1 \otimes U_2 \to W$:
f1 = Array[a, {5, 10}]
f2 = Array[b, {5, 8}]
Mathematically, the easiest way to represent this is a tensor product + contraction:
TensorContract[op \[TensorProduct] f1 \[TensorProduct] f2, {{2, 4}, {3, 6}}]
However, this is horrendously slow. In reality, I'm working with operators between much larger spaces, and I want to compose several of them, for example something like $O \circ (F_1 \otimes F_2 \otimes F_3) \circ \dots$ where there are maybe three nested levels of composition. So, speed matters.
A much faster way is to use the dot product and transpositions:
Transpose[Transpose[op . f2, 2 <-> 3] . f1, 2 <-> 3]
The speed difference is huge:
RepeatedTiming[TensorContract[op\[TensorProduct]f1\[TensorProduct]f2, {{2, 4}, {3, 6}}];]
(* {5.16933, Null} *)
RepeatedTiming[Transpose[Transpose[op . f2, 2 <-> 3] . f1, 2 <-> 3];]
(* {0.000972633, Null} *)
This is to be expected: TensorProduct constructs a humongous tensor, only to discard most of it immediately.
However, I find the second solution a bit ugly. I don't like to write the transpositions twice, for example. And figuring out the indices for the transposition becomes a mess when $F_1, F_2\dots$ are themselves higher-rank tensor.
Is there a nicer, built-in way to express this tensor dot product / contraction that's a bit more intuitive, while remaining efficient? Bonus point if it works with sparse arrays (the tensors I work with are huge but very sparse). In clear language, given tensors $T_1$ of rank $(d_1, \dots, d_k)$, $T'$ of rank $(d'_1, \dots, d'_l)$, and an index $i$ such that $d_i = d'_1$, I want a new tensor of rank $(d_1, \dots, d_{i-1}, d'_2, \dots, d'_l, d_{i+1}, \dots, d_k)$.
