Matrix Multiplication in context of row and column vectors

Question

I've been looking at some matrices in Mathematica and I've noticed something very weird: they're extremely temperamental when it comes to dot products! For example, if I have the following,

i = {{1}, {2}, {3}}
j = {1, 2, 3}
k = i.j

then as far as I remember I should get a $3\times 3$ matrix since we have (3×1).(1×3) and the 1;s "cancel" to produce $3\times 3$. But I get this error:

Dot::dotsh: "Tensors {{1},{2},{3}} and {1,2,3} have incompatible shapes. "

But then if I do j.i I get {14} which is correct. Also, I've noticed sometimes with more complex matrices i and j, that if I set k = i.j, no calculation will be done, and the output will just show the two matrices in juxtaposition. But if I just do i.j (without the "k=" part), a calculated output is shown. Does anyone know what the heck I'm doing wrong? This is driving me crazy!

Answer 1

This is in fact tricky. But j is not what it looks.

TensorRank[i]

gives 2 and its dimensions are {3, 1}. j is different:

TensorRank[j]

gives 1 and its dimensions are {3} instead of {3, 1}. A fix.

j = {{1, 2, 3}}

and you get

i.j

{{1, 2, 3}, {2, 4, 6}, {3, 6, 9}}

j.i

gives {{14}}.

The reason it apparently works with j.i is that in this case Mathematica applies vector . matrix. What you wanted makes sense when dealing with tensors of the same rank. A visit to this Mathematica guide may help.

Answer 2

Mathematica does not have the concept of row or column vectors like you may be used to. The concept isn't really necessary either and is just a convention to visualize the dot product (although I know there are people that vehemently object to this statement).

In dot products like $M\cdot\vec{x}$ and $\vec{x}^{^\top}\cdot M$ Mathematica uses $\vec{x}$ as your column and row vector automatically. Both are entered in Mathematica as the same list.

{{1, 2}, {3, 4}}.{5, 6}
(* {17, 39} *)

{5, 6}.{{1, 2}, {3, 4}}
(* {23, 34} *)

Of course, this means that the length of the vector must be equal to the number of columns in the matrix in the former case, and to the number of rows in the latter case.

So, this doesn't work (2x3 matrix):

{{1, 2, 1}, {3, 4, 1}}.{5, 6}

During evaluation of Dot::dotsh: Tensors {{1,2,1},{3,4,1}} and {5,6} have incompatible shapes. >>

(* {{1, 2, 1}, {3, 4, 1}}.{5, 6} *)

and this works (3x2 matrix):

{{1, 2}, {3, 4}, {1, 1}}.{5, 6}
(* {17, 39, 11} *)

If you insist on using column and row vectors, you can mimic them by using similarly looking matrices. The matrix equivalent of a column vector would be {{x1}, {x2},...,{xn}}. The equivalent of a row vector would be {{x1, x2,...,xn}}. If you use MatrixForm or TableForm they display just like column and row vectors would, except that they are still matrices in disguise (nx1 and 1xn ones, respectively). Those matrices behave in dot products just like their row and column vector brethren would in languages that have them, but remember, they are not necessary at all in Mathematica.

{{1, 2}, {3, 4}}.{{5}, {6}}
(* {{17}, {39}} *)

{{5, 6}}.{{1, 2}, {3, 4}}
(* {{23, 34}} *)

The above code typeset in TraditionalForm (just Ctrl+Alt+T of the above lines) displays as:

The main disadvantage of this technique is having to deal with extra curly brackets that have to be removed when you want to use the resulting vector as a list, and the fact that you have to change the brackets depending on the position in the dot product. Additionally, there are functions that expect Mathematica vectors (simple lists) that won't work when provided with their matrix emulations.

Answer 3

Another way of keeping track of vector/matrix products is with the function Outer, which calculates the outer product of two lists. For example:

i = {3, 4, 5};
j = {1, 2, 3};
Outer[Times, i, j]
{{3, 6, 9}, {4, 8, 12}, {5, 10, 15}}

There is also the corresponding function Inner for inner products:

Inner[Times, i, j, Plus]
26

Answer 4

This isn't as mysterious or as unexpected as it seems. In common mathematical usage, vectors (the algebraic ones we learn about in school, not necessarily the abstract objects) are lists, but they can be represented as matrices (sometimes confusingly called a column vector).

Matrix multiplication is defined between two matrices, and simply treats a right-hand vector argument as its matrix representation, and a left-hand vector argument as the transpose of that representation. The result of either multiplication is a vector.

Mathematica's . simply implements this convention, while MatrixForm of a vector simply shows its matrix representation.

Answer 5

I totally agree with @InquisitiveInquirer. Multiplication of a (3x1) with a (1x3) matrix should give a (3x3) matrix. I was struggling with this as well. I worked with Mathematica for years, but suddenly I need to switch to Mathlab, because it makes better sense with matrix multiplication.

In case you'd like to switch to Mathlab, I give the comands + MWE for (2x1) and (1x2) matrix multiplication below.

b1 = {a, b};
b2 = {c, d};
Print["Mathematica ", MatrixForm[b1], "*", MatrixForm[b2], " = ", MatrixForm[b1*b2], ". In Mathlab: ", MatrixForm[b1], ".*", MatrixForm[b2], " = ", MatrixForm[b1*b2]]

$$\text{Mathematica }\left(\begin{array}{c}a\\b\end{array}\right)*\left(\begin{array}{c}c\\d\end{array}\right)\text{ = }\left(\begin{array}{c}a c \\b d\end{array}\right)\text{. In Mathlab: }\left(\begin{array}{c}a\\b\end{array}\right).*\left(\begin{array}{c}c\\d\end{array}\right)\text{ = }\left(\begin{array}{c}a c \\b d\end{array}\right) $$

Print["Mathematica ", MatrixForm[b1], ".", MatrixForm[b2], " = ", b1.b2, ". In Mathlab: ", MatrixForm[{b1}], "*", MatrixForm[b2], " = ", b1.b2]

$$ \text{Mathematica }\left(\begin{array}{c} a \\ b \end{array}\right) . \left(\begin{array}{c} c \\ d\end{array}\right) = a c+b d\text{. In Mathlab: }\left(\begin{array}{cc} a & b\end{array}\right) * \left(\begin{array}{c} c \\ d\end{array}\right) = a c+b d$$

Print["Mathematica Outer[Times,{{c,d}},{a,b}] = ",MatrixForm[Outer[Times,{b2},b1]],". In Mathlab: ",MatrixForm[b1],"*",MatrixForm[{b2}]," = ",MatrixForm[Outer[Times,{b2},b1]]]

$$\text{Mathematica Outer[Times,}\left(\begin{array}{cc}c & d\end{array}\right),\left(\begin{array}{c}a\\b\end{array}\right) \text{] = }\left(\begin{array}{c}a c & b c \\a d & b d \end{array}\right)\text{. In Mathlab: }\left(\begin{array}{c}a\\b\end{array}\right)*\left(\begin{array}{cc}c & d\end{array}\right) = \left(\begin{array}{c}a c & b c \\a d & b d \end{array}\right) $$