If we have nested matrices, such as:
$$ \begin{bmatrix} \begin{bmatrix} a_{1,1} & a_{1,2}\\ a_{2,1} & a_{2,2}\\ \end{bmatrix} \begin{bmatrix} b_{1,1} & b_{1,2}\\ b_{2,1} & b_{2,2}\\ \end{bmatrix} \\ \begin{bmatrix} c_{1,1} & c_{1,2}\\ c_{2,1} & c_{2,2}\\ \end{bmatrix} \begin{bmatrix} d_{1,1} & d_{1,2}\\ d_{2,1} & d_{2,2}\\ \end{bmatrix} \\ \end{bmatrix} $$
...and we wish to multiply two of these matrices together, how can we accomplish this?
Here I assume that the conventional rules of matric multiplication apply. For example, we treat each "sub-matrix" (or block matrix, if you prefer), for lack of a better term, as an entry in the main matrix. Then we recursively apply the rules of matrix multiplication.
AN ATTEMPT TO EXPLAIN
So, for example, the matrix from above could be represented as:
$$ \begin{bmatrix} a & b\\ c & d\\ \end{bmatrix} $$
...and multiplied by another matrix to get:
$$ \begin{bmatrix} a & b\\ c & d\\ \end{bmatrix} * \begin{bmatrix} e & f\\ g & h\\ \end{bmatrix} = \begin{bmatrix} (ae+bg) & (af+bh)\\ (ce+dg) & (cf+dh)\\ \end{bmatrix} $$
Where each entry above is actually another matrix. For example, the entry $(ae+bg)$ is actually:
$$ (ae+bg) = \begin{bmatrix} a_{1,1} & a_{1,2}\\ a_{2,1} & a_{2,2}\\ \end{bmatrix} * \begin{bmatrix} e_{1,1} & e_{1,2}\\ e_{2,1} & e_{2,2}\\ \end{bmatrix} + \begin{bmatrix} b_{1,1} & b_{1,2}\\ b_{2,1} & b_{2,2}\\ \end{bmatrix} * \begin{bmatrix} g_{1,1} & g_{1,2}\\ g_{2,1} & g_{2,2}\\ \end{bmatrix} $$
Dot
can operate on any pair of tensors (for as long as their sizes match). It will contract the last index of the first tensor with the first index of the last tensor.TensorContract
allows contracting arbitrary indices. $\endgroup$