# Multiple Simulations of a Markov Chain

I am trying to create a simple series of Bernoulli bits whose probabilities follow a two-state Markov chain.

ProbInitial = 1 (*Initial State of Markov Chain*)
(*Define Two-state Markov Transition Matrix*)
p11 = 0.4;(*Probability of a Success given a Prior Success*)
p01 = 0.8;(*Probability of a Success given a Prior Failure*)


I want to create a Table containing NEvents for each of NSimulations.

I am able to create a single Table containing NEvents that behaves properly, i.e., the estimates of p01 and p11 are correct.

However, when I try to scale the following code to NSimulations, I get strange values of p011 and p11 back for all the NSimulations on some runs. Sometimes I get the same sequence of bits for all the NSimulations.

I realized that the Mathematica was remembering function values and decided to use the Module function as follows:

ProbSuccess[ii_Integer] := If[ii == 0, p01, p11]

ClearAll[SimDataUnit];
SimDataUnit[k_] := Module[
{Sucess},
Success[1] := RandomVariate[BernoulliDistribution[ProbInitial]];
Success[ii_Integer] := Success[ii] =
RandomVariate[BernoulliDistribution[ProbSuccess[Success[ii - 1]]]];
Table[Success[jj], {jj, 1, NEvents}]
]


I call SimDataUnit[k] to to get different random realizations for each of the Nsimulations.

SimDataMany = Table[SimDataUnit[ii], {ii, 1, NSimulations}]


I was hoping that that making the {Success} variable local to the Module would make a difference but it did not. Any and all suggestions are welcome.

-
You misspelled Success in the local declaration... – rogerl Apr 3 '14 at 12:41

Why not just take advantage of built-in, faster means? E.g. to create say five streams of "bits" each with specified transition probabilities:

RandomFunction[DiscreteMarkovProcess[1, {{.2, .8}, {.6, .4}}], {0, 20}, 5]["States"] - 1

(*

{{0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0},
{0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0},
{0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1},
{0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 0, 1},
{0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1}}

*)

-
Thanks. That exactly what I needed. I am on Mathematica 8 for the Macintosh. I will look for an upgrade to Mathematica 9. It seems RandomFunction is new to Mathematica 9, which would explain why it did not show up on Function Search. – Murali Apr 2 '14 at 11:42