How do I change the initial state of a discrete Markov process?

Question

I have a simple Markov process defined with

mp = DiscreteMarkovProcess[1, ( {
     {1/3, 1/3, 1/3, 0},
     {1/2, 1/4, 0, 1/4},
     {0, 0, 1, 0},
     {0, 0, 0, 1}
    } )];

How I change the initial state of this process, without defining an entirely new process. Is there a way to "set" the initial probabilities of mp to different values?

Answer 1

Update: An alternative approach is to use "TransitionMatrix" of the original process to create a new process with the desired initial state:

mp = DiscreteMarkovProcess[1, 
  {{1/3, 1/3, 1/3, 0}, {1/2, 1/4, 0, 1/4}, {0, 0, 1, 0}, {0, 0,  0, 1}}];
MarkovProcessProperties[mp, "InitialProbabilities"]

1

mp = DiscreteMarkovProcess[2, MarkovProcessProperties[mp, "TransitionMatrix"]];
MarkovProcessProperties[mp, "InitialProbabilities"]

2

---

Original answer:

Resetting the first Part of mp, e.g., mp[[1]]=2 seems to work.

mp = DiscreteMarkovProcess[1, ({{1/3, 1/3, 1/3, 0}, {1/2, 1/4, 0, 1/4}, {0, 0, 1, 0},
    {0, 0,  0, 1}})];
Row[{MarkovProcessProperties[mp] /. Grid[x_, y__] :> 
            Grid[x[[Join[Range[4], Range[-8, -1]]]], y], 
    mp[[1]] = 2; MarkovProcessProperties[mp] /. Grid[x_, y__] :> 
            Grid[x[[Join[Range[4], Range[-8, -1]]]], y]},  Spacer[5]]

Answer 2

There's no feature of DiscreteMarkovProcess that supports this, but you can accomplish what you want using basic language features, for example:

mp = DiscreteMarkovProcess[state, ( {
     {1/3, 1/3, 1/3, 0},
     {1/2, 1/4, 0, 1/4},
     {0, 0, 1, 0},
     {0, 0, 0, 1}} )];

Block[{state=3}, <do stuff with MP>...]

Of course, you need to insure that whatever symbol you choose to use is not defined at the time of the DMP definition, otherwise that value will be used in the definition.

Answer 3

The question really boils down to the more general question: “Can I modify an existing Mathematica expression without building a new one?” Yes and no; it’s a bit of a discouraged practice since there may be unanticipated side effects. But since Mathematica expressions are just lists of lists, you can take parts of an existing expression and recombine them in new ways to create variants.

Your code:

mp = DiscreteMarkovProcess[1, ( {
     {1/3, 1/3, 1/3, 0},
     {1/2, 1/4, 0, 1/4},
     {0, 0, 1, 0},
     {0, 0, 0, 1}
    } )];

You could also use the "TransitionMatrix" property of the MarkovProcessProcess function to do the same thing, as the Update to the question shows.

Notice that you can extract or set the state as mp[[1]] and you can extract or set the transition rate matrix as mp[[2]]; see the Part function in Mathematica if you are not familiar with this syntax for list items. Although Part is usually used with pure lists, it is applicable to any Mathematica object or expression.

So we can build a new Markov process using state #2 as the initial state, for example, like this:

mp2 = DiscreteMarkovProcess[2, mp[[2]]];

Or as you discovered, you could use mp[[1]]=2 to set a new initial state within mp. That’s allowed, but in general you don’t want to change existing items as the side effects produced may be harder to understand in future calculations. Memory is cheap, so I would find it preferable to maintain distinct mp and mp2 (or mp1 and mp2) objects. This also dates back to the days of Lisp, which Mathematica inherits from, where side effects in code were generally discouraged unless absolutely needed.

The best approach, if you knew beforehand that you might want to change the initial state, is to simply define the transition rate matrix first, independently. Then you can use the matrix to build any number of DiscreteMarkovProcess objects. So:

m = {
     {1/3, 1/3, 1/3, 0},
     {1/2, 1/4, 0, 1/4},
     {0, 0, 1, 0},
     {0, 0, 0, 1}
    };
mp = DiscreteMarkovProcess[1,m];
mp2 = DiscreteMarkovProcess[2,m];

Or a sneakier way to have done this would be along these lines, “capturing” the matrix in the first definition as m, and then using it in the second. This is perfectly legal, though quite a bit harder to read. Generally speaking, you want to keep side effects of evaluation to a minimum, so this is a bit uglier, certainly.

mp = DiscreteMarkovProcess[1, m=( {
     {1/3, 1/3, 1/3, 0},
     {1/2, 1/4, 0, 1/4},
     {0, 0, 1, 0},
     {0, 0, 0, 1}
    } )];
mp2 = DiscreteMarkovProcess[2,m];

Hope this info helps!