# Tallying over columns of matrix

Suppose I have a $$300 \times 100$$ matrix, with entries 0,1 or 2. The first row represents the initial state of a 100-parameter discrete dynamical system (each parameter taking only those three values) and each subsequent row represents the state of the system after one timestep, for 300 steps.

I would like to construct a function that takes this matrix and a column $$j$$, and returns the pair probabilities $$p^{(j)}_{xy}$$ of observing states $$x, y$$ in subsequent time steps. I want to do this by tallying the events "$$x$$ then $$y$$" then normalizing. How can I do this?

• Could you please describe event x and event y exactly?
– Syed
May 1 at 3:42
• x and y are parameter states, i.e elements in $\{0,1,2\]$ The event "1 then 2' just means that in my column the string "12" occurs
– asdf
May 1 at 3:46
• And when you say  ... construct a function that takes this matrix and a column j, do you mean the 300x100 matrix operates on a 100x1 vector?
– Syed
May 1 at 3:49
• No. I may in future want to operate on different matrices, so I'd like a function that takes as arguments a matrix $M$ and a number $j$ that will be the column number, and outputs these pair probabilities for that column of the matrix.
– asdf
May 1 at 4:06
• In what way is the answer put forward by @lericr not sufficient for your needs?
– Syed
May 1 at 7:08

Not sure what would be the best representation in your case, but here's a possible way to start. Let's start with a simplified case of one isolated column (as a List):

columnData = {0, 2, 1, 1, 2, 2, 1, 2, 0, 1}
(* this was generated randomly, but working with an explicit set of data will be clearer *)


Since you want pairs, we can create the pairwise data:

pairs = Partition[columnData, 2, 1]
(* {{0, 2}, {2, 1}, {1, 1}, {1, 2}, {2, 2}, {2, 1}, {1, 2}, {2, 0}, {0, 1}} *)


One thing we can do at this point is create an EmpiricalDistribution:

pairDist = EmpiricalDistribution[pairs]


Alternatively, you might Tally the pairs and create WeightedData:

pairWeighted = WeightedData @@ Transpose[Tally[pairs]]


At this point, I don't know where you want to go. But we can back up and figure out how to extract a column from your matrix:

fetchColumn[colIdx_, matrix_] := matrix[[All, colIdx]]


Now, you can create another function if you want to fetch the column and then apply whichever of the above procedures you want.

EDIT WITH FURTHER SUGGESTIONS

With pairWeighted, you can see the probabilities associated with each pair like this:

pairWeighted["EmpiricalPDF"]


As an aside, with either pairWeighted or pairDist you can see what properties are available like this:

pairDist["Properties"]
pairWeighted["Properties"]


To get a probability function:

PDF[pairDist] (* a pure function *)
PDF[pairDist][{0, 0}] (* returns the probability of the pair {0,0}*)


Once you have a PDF, there are many other properties/functions you can apply. Again, I don't know where you're going with this, so I can only suggest you read the documentation.

Of course, all of this might have been overkill. If you just want "tallying" and "normalizing", then you could have just done your analysis with Tally[pairs] or Counts[pairs] (to normalize, divide by the number of pairs in the column, which I assume will be 299).

Caveat: I'm not a statistician, so maybe my assumptions and inferences were not the "natural" ones.

• Thanks for responding. But how do I proceed after creating an EmpiricalDistribution or WeightedData? What function do I apply to pairDist or pairWeighted?
– asdf
May 1 at 14:17
• I updated my answer May 1 at 14:51