Tallying over columns of matrix

Question

Suppose I have a $300 \times 100$ matrix, with entries 0,1 or 2. The first row represents the initial state of a 100-parameter discrete dynamical system (each parameter taking only those three values) and each subsequent row represents the state of the system after one timestep, for 300 steps.

I would like to construct a function that takes this matrix and a column $j$, and returns the pair probabilities $p^{(j)}_{xy}$ of observing states $x, y$ in subsequent time steps. I want to do this by tallying the events "$x$ then $y$" then normalizing. How can I do this?

Answer 1

Not sure what would be the best representation in your case, but here's a possible way to start. Let's start with a simplified case of one isolated column (as a List):

columnData = {0, 2, 1, 1, 2, 2, 1, 2, 0, 1} 
(* this was generated randomly, but working with an explicit set of data will be clearer *)

Since you want pairs, we can create the pairwise data:

pairs = Partition[columnData, 2, 1]
(* {{0, 2}, {2, 1}, {1, 1}, {1, 2}, {2, 2}, {2, 1}, {1, 2}, {2, 0}, {0, 1}} *)

One thing we can do at this point is create an EmpiricalDistribution:

pairDist = EmpiricalDistribution[pairs]

Alternatively, you might Tally the pairs and create WeightedData:

pairWeighted = WeightedData @@ Transpose[Tally[pairs]]

At this point, I don't know where you want to go. But we can back up and figure out how to extract a column from your matrix:

fetchColumn[colIdx_, matrix_] := matrix[[All, colIdx]]

Now, you can create another function if you want to fetch the column and then apply whichever of the above procedures you want.

EDIT WITH FURTHER SUGGESTIONS

With pairWeighted, you can see the probabilities associated with each pair like this:

pairWeighted["EmpiricalPDF"]

As an aside, with either pairWeighted or pairDist you can see what properties are available like this:

pairDist["Properties"]
pairWeighted["Properties"]

To get a probability function:

PDF[pairDist] (* a pure function *)
PDF[pairDist][{0, 0}] (* returns the probability of the pair {0,0}*)

Once you have a PDF, there are many other properties/functions you can apply. Again, I don't know where you're going with this, so I can only suggest you read the documentation.

Of course, all of this might have been overkill. If you just want "tallying" and "normalizing", then you could have just done your analysis with Tally[pairs] or Counts[pairs] (to normalize, divide by the number of pairs in the column, which I assume will be 299).

Caveat: I'm not a statistician, so maybe my assumptions and inferences were not the "natural" ones.