# Matrix Multiplication in context of row and column vectors

I've been looking at some matrices in Mathematica and I've noticed something very weird: They're extremely temperamental when it comes to dot products! For example, if I have the following,

i={{1},{2},{3}}
j={1,2,3}
k=i.j


the as far as I remember I should get a $3x3$ matrix since we have (3x1).(1x3) and the 1s "cancel" to produce 3x3. But I get this error:

Dot::dotsh: "Tensors {{1},{2},{3}} and {1,2,3} have incompatible shapes. "


But then if I do j.i I get {14} which is correct. Also, I've noticed sometimes with more complex matrices i and j, that if I set k=i.j, no calculation will be done, and the output will just show the two matrices in juxtaposition. But if I just do i.j (without the "k=" part), a calculated output is shown. Does anyone know what the heck I'm doing wrong? This is driving me crazy!

-
To make it work: j={{1,2,3}}. You can check then dimensions with Dimensions[j] -> {1,3}. – Kuba Jan 13 '14 at 22:34
That is not an 1 by 3 matrix but a length 3 vector. Mathematica works with general tensors of arbitrary dimensions, not only 2-dimensional matrices like many MATLAB-inspired systems. Mathematica supports true 1-dimensional vectors. This is described on the doc page of Dot, take a look under Details. – Szabolcs Jan 13 '14 at 22:55

This is in fact tricky. But j is not what it looks.

TensorRank[i]


gives 2 and its dimensions are {3,1}. j is different:

TensorRank[j]


gives 1 and its dimensions are {3} instead of {3,1}. A fix.

j = {{1,2,3}}


and you get

i.j


{{1, 2, 3}, {2, 4, 6}, {3, 6, 9}}

j.i


gives {{14}}.

The reason it apparently works with j.i is that in this case Mathematica applies vector . matrix. What you wanted makes sense when dealing with tensors of the same rank. A visit to this Mathematica guide may help.

-

Mathematica does not have the concept of row or column vectors like you may be used to. The concept isn't really necessary either and is just a convention to visualize the dot product (although I know there are people that vehemently object to this statement).

In dot products like $M\cdot\vec{x}$ and $\vec{x}^{^\top}\cdot M$ Mathematica uses $\vec{x}$ as your column and row vector automatically. Both are entered in Mathematica as the same list.

{{1, 2}, {3, 4}}.{5, 6}
(* {17, 39} *)

{5, 6}.{{1, 2}, {3, 4}}
(* {23, 34} *)


Of course, this means that the length of the vector must be equal to the number of columns in the matrix in the former case, and to the number of rows in the latter case.

So, this doesn't work (2x3 matrix):

{{1, 2, 1}, {3, 4, 1}}.{5, 6}


During evaluation of Dot::dotsh: Tensors {{1,2,1},{3,4,1}} and {5,6} have incompatible shapes. >>

(* {{1, 2, 1}, {3, 4, 1}}.{5, 6} *)


and this works (3x2 matrix):

{{1, 2}, {3, 4}, {1, 1}}.{5, 6}
(* {17, 39, 11} *)


If you insist on using column and row vectors, you can mimic them by using similarly looking matrices. The matrix equivalent of a column vector would be {{x1}, {x2},...,{xn}}. The equivalent of a row vector would be {{x1, x2,...,xn}}. If you use MatrixForm or TableForm they display just like column and row vectors would, except that they are still matrices in disguise (nx1 and 1xn ones, respectively). Those matrices behave in dot products just like their row and column vector brethren would in languages that have them, but remember, they are not necessary at all in Mathematica.

{{1, 2}, {3, 4}}.{{5}, {6}}
(* {{17}, {39}} *)

{{5, 6}}.{{1, 2}, {3, 4}}
(* {{23, 34}} *)


The above code typeset in TraditionalForm (just Ctrl+Alt+T of the above lines) displays as:

The main disadvantage of this technique is having to deal with extra curly brackets that have to be removed when you want to use the resulting vector as a list, and the fact that you have to change the brackets depending on the position in the dot product. Additionally, there are functions that expect Mathematica vectors (simple lists) that won't work when provided with their matrix emulations.

-
But the matrix form of {5,6} is the same as {{5}, {6}}. So does that mean Mathematica treat list as column vector? – kww Jun 13 at 4:56
How Mathematica treats those two is explained above. Please read carefully. – Sjoerd C. de Vries Jun 13 at 13:25

Another way of keeping track of vector/matrix products is with the function Outer, which calculates the outer product of two lists. For example:

i = {3, 4, 5};
j = {1, 2, 3};
Outer[Times, i, j]
{{3, 6, 9}, {4, 8, 12}, {5, 10, 15}}


There is also the corresponding function Inner for inner products:

Inner[Times, i, j, Plus]
26

-

This isn't as mysterious or as unexpected as it seems. In common mathematical usage, vectors (the algebraic ones we learn about in school, not necessarily the abstract objects) are lists, but they can be represented as matrices (sometimes confusingly called a column vector).

Matrix multiplication is defined between two matrices, and simply treats a right-hand vector argument as its matrix representation, and a left-hand vector argument as the transpose of that representation. The result of either multiplication is a vector.

Mathematica's . simply implements this convention, while MatrixForm of a vector simply shows its matrix representation.

-