I'm simulating a discrete time Markov process on a finite state space corresponding to a linear interval $(x_1, x_2, ..., x_N) \in X$, where each time step $t_i$ involves updating the state of some number of agents labeled "Red" and "Blue". At any time point $t_i$ I have an array that keeps track of the positions and colors of the agents along the interval:

agentArray[[t]] = {
    {"Blue", 9}, {"Red", 8}, {"Blue", 9}, {"Blue", 9}, {"Blue", 8}, {"Blue", 8},
    {"Blue", 8}, {"Blue", 6}, {"Blue", 9}, {"Blue", 9}, {"Blue", 8}, {"Blue", 8},
    {"Red", 7}, {"Blue", 8}, {"Blue", 8}, {"Blue", 3}};

I'd like to first and foremost have a nice way in which to illustrate the occupancy levels (for agents of the types "Red" and "Blue") of the discrete states $1$ through $N$ in the Markov chain at some time point $t_i$. Given that Wolfram Research recently extensively updated their Markov processes toolkit in Mathematica, is there a nice way to do this automatically? Something like a heat map superimposed on a graph representation of the linear interval, with explicitly stated counts for the number of red and blue agents at each vertex?

Secondly, I was wondering if there was a way to make an animation showing the evolution of the Markov process? Something like the aforementioned graphic where we sample the state of the Markov chain at every $k$th time point?

Specifically: How do we use the aforementioned data structure to show a graph, with heat/color encoded "Red" and "Blue" occupancy levels at each vertex, with the occupancy counts also explicitly written in text above each vertex?


1 Answer 1


I present this for motivation. This is a small toy example,

The following image was created with Mathematica prior to the current implementations. (post here)

enter image description here

Using random walk on a graph as inspiration the following is presented:

  1. Transition probabilities:

    tp = {{0, 1/2, 1/2, 0}, {1/2, 0, 1/2, 0}, {1/3, 1/3, 0, 1/3}, {1, 0, 
    0, 0}}
  2. Create Markov objects and random function (in this case 10 steps)

    mkv = DiscreteMarkovProcess[{1, 0, 0, 0}, tp];
    rf = RandomFunction[mkv, {0, 10}, 25];
  3. Note that 25 paths are created to simulate. Now the paths are extracted and the number of elements in each state counted:

    paths = #[[All, 2]] & /@ rf["Paths"];
    cnt = Map[Function[x, Count[#, x]], Range[4]] & /@ Transpose@paths;
  4. Now to generate simulation from paths:

    bc = BarChart[#, ChartLabels -> Range[4], PlotRange -> {0, 25}, 
     LabelingFunction -> Above, PlotRangePadding -> {0, 5}] & /@ cnt;
    anim = Transpose@paths;
    arp = ArrayPlot[Partition[#, 5], 
     ColorRules -> {1 -> Red, 2 -> Blue, 3 -> Green, 4 -> Yellow}] & /@
    anex = MapThread[
        Column[{Style["Step " <> ToString[#3], 
           FontFamily -> "Calibri", 12], #2}]}], Graph[mkv]}, 
     Spacings -> 0] &, {arp, bc, Range[0, 10]}];

This was used to create the animated gif that steps through each path. Each element of array changes color to relevant state ( {1 -> Red, 2 -> Blue, 3 -> Green, 4 -> Yellow}). I should have put a legend but this is for illustration purposes:

enter image description here

This may not be exactly what you want but i hope it allows you to achieve it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.