# Simulating jump Markov process

I am trying to simulate a jump Markov process as follows. I define the process $$N_j = N_{j-1} + 1$$ with probability 1/2 or $$N_j = N_{j-1} - 1$$ with probability $$1/2,$$ with $$N_j = 0$$ as an absorbing state. I tried

p = DiscreteMarkovProcess[{0, 1}, {{1/2, 1/2}, {1/2, 1/2}}];
data = RandomFunction[p, {0, 50}]
ListPlot[data, Filling -> Axis, Ticks -> {Automatic, {0, 1, 2}}]


but I am not getting any results that are in the absorbing state. Could someone help clarify this? I feel I am not understanding this...

• Your Markov chain matrix does not correspond to the formulas you gave. If we assume that the "corner" states switch to their adjacent states with probability 1, then the matrix should be something like {{0, 1, 0}, {1/2, 0, 1/2}, {0, 1, 0}}. But then you do not have any absorbing states. This (very similar) matrix has an absorbing state: {{0, 1, 0}, {1/2, 0, 1/2}, {0, 0, 1}}. May 26, 2021 at 12:49

Make the matrix:

nStates = 6;
mat = DiagonalMatrix[ConstantArray[1/2, nStates - 1], 1] +
DiagonalMatrix[ConstantArray[1/2, nStates - 1], -1];
mat = #/Total[#] & /@ mat; (*Sum normalize per row*)
mat[[-1]] = ConstantArray[0, nStates]; mat[[-1, -1]] = 1;(*Make the last state an absorbing state*)
MatrixForm[mat]


Simulate:

SeedRandom[13];
p = DiscreteMarkovProcess[mat[[1]], mat];
data = RandomFunction[p, {0, 50}]
ListPlot[data, Filling -> Axis, Ticks -> {Automatic, {0, 1, 2}}]


Interactive interface to visualize the simulation:

path = data["Values"];
Manipulate[
Graph[p, GraphHighlight -> path[[time + 1]], GraphHighlightStyle -> "VertexConcaveDiamond", PlotLabel -> "Time \[LongEqual] " <> ToString[time], ImageSize -> Medium], {time, 0, Length[path] - 1, 1}, SaveDefinitions -> True]