# How do I change the initial state of a discrete Markov process?

I have a simple Markov process defined with

mp = DiscreteMarkovProcess[1, ( {
{1/3, 1/3, 1/3, 0},
{1/2, 1/4, 0, 1/4},
{0, 0, 1, 0},
{0, 0, 0, 1}
} )];


How I change the initial state of this process, without defining an entirely new process. Is there a way to "set" the initial probabilities of mp to different values?

• Your question is unclear: the initial state is... a state, not a probability. Is that what you want to change, or do you mean you want to change the transition matrix without redefining it? If the latter, just make the matrix symbolic, and then enclose your cases in Block, with the symbols defined to the values you want. – ciao May 10 '14 at 22:17
• The first argument can be either a single state or a list of initial state probabilities. You can simply define mp as a function having the initial state or state vector as one of its arguments. – Sjoerd C. de Vries May 10 '14 at 22:46
• @SjoerdC.deVries: That sounds like the answer (if in fact there's no particular way to change the initial state once it's been set. – orome May 11 '14 at 2:27
• @raxacoricofallapatorius: You can use the same technique I allude to for initial state, just define the state as a symbol, easy peasy. – ciao May 11 '14 at 4:48
• @rasher: If you change the answer to match the question (about initial state, not the transition matrix), that will work. – orome May 12 '14 at 16:44

Update: An alternative approach is to use "TransitionMatrix" of the original process to create a new process with the desired initial state:

mp = DiscreteMarkovProcess[1,
{{1/3, 1/3, 1/3, 0}, {1/2, 1/4, 0, 1/4}, {0, 0, 1, 0}, {0, 0,  0, 1}}];
MarkovProcessProperties[mp, "InitialProbabilities"]


1

mp = DiscreteMarkovProcess[2, MarkovProcessProperties[mp, "TransitionMatrix"]];
MarkovProcessProperties[mp, "InitialProbabilities"]


2

Resetting the first Part of mp, e.g., mp[[1]]=2 seems to work.

mp = DiscreteMarkovProcess[1, ({{1/3, 1/3, 1/3, 0}, {1/2, 1/4, 0, 1/4}, {0, 0, 1, 0},
{0, 0,  0, 1}})];
Row[{MarkovProcessProperties[mp] /. Grid[x_, y__] :>
Grid[x[[Join[Range[4], Range[-8, -1]]]], y],
mp[[1]] = 2; MarkovProcessProperties[mp] /. Grid[x_, y__] :>
Grid[x[[Join[Range[4], Range[-8, -1]]]], y]},  Spacer[5]]


There's no feature of DiscreteMarkovProcess that supports this, but you can accomplish what you want using basic language features, for example:

mp = DiscreteMarkovProcess[state, ( {
{1/3, 1/3, 1/3, 0},
{1/2, 1/4, 0, 1/4},
{0, 0, 1, 0},
{0, 0, 0, 1}} )];

Block[{state=3}, <do stuff with MP>...]


Of course, you need to insure that whatever symbol you choose to use is not defined at the time of the DMP definition, otherwise that value will be used in the definition.

The question really boils down to the more general question: “Can I modify an existing Mathematica expression without building a new one?” Yes and no; it’s a bit of a discouraged practice since there may be unanticipated side effects. But since Mathematica expressions are just lists of lists, you can take parts of an existing expression and recombine them in new ways to create variants.

mp = DiscreteMarkovProcess[1, ( {
{1/3, 1/3, 1/3, 0},
{1/2, 1/4, 0, 1/4},
{0, 0, 1, 0},
{0, 0, 0, 1}
} )];


You could also use the "TransitionMatrix" property of the MarkovProcessProcess function to do the same thing, as the Update to the question shows.

Notice that you can extract or set the state as mp[[1]] and you can extract or set the transition rate matrix as mp[[2]]; see the Part function in Mathematica if you are not familiar with this syntax for list items. Although Part is usually used with pure lists, it is applicable to any Mathematica object or expression.

So we can build a new Markov process using state #2 as the initial state, for example, like this:

mp2 = DiscreteMarkovProcess[2, mp[[2]]];


Or as you discovered, you could use mp[[1]]=2 to set a new initial state within mp. That’s allowed, but in general you don’t want to change existing items as the side effects produced may be harder to understand in future calculations. Memory is cheap, so I would find it preferable to maintain distinct mp and mp2 (or mp1 and mp2) objects. This also dates back to the days of Lisp, which Mathematica inherits from, where side effects in code were generally discouraged unless absolutely needed.

The best approach, if you knew beforehand that you might want to change the initial state, is to simply define the transition rate matrix first, independently. Then you can use the matrix to build any number of DiscreteMarkovProcess objects. So:

m = {
{1/3, 1/3, 1/3, 0},
{1/2, 1/4, 0, 1/4},
{0, 0, 1, 0},
{0, 0, 0, 1}
};
mp = DiscreteMarkovProcess[1,m];
mp2 = DiscreteMarkovProcess[2,m];


Or a sneakier way to have done this would be along these lines, “capturing” the matrix in the first definition as m, and then using it in the second. This is perfectly legal, though quite a bit harder to read. Generally speaking, you want to keep side effects of evaluation to a minimum, so this is a bit uglier, certainly.

mp = DiscreteMarkovProcess[1, m=( {
{1/3, 1/3, 1/3, 0},
{1/2, 1/4, 0, 1/4},
{0, 0, 1, 0},
{0, 0, 0, 1}
} )];
mp2 = DiscreteMarkovProcess[2,m];


Hope this info helps!