# State “i” goes to state “j”: list accessible states in a Markov-chain

I would like to have a code that given a state "i" in S, it give the set of states "j" which are accessible from "i".

Is that possible in mathematica?

My state space (S) is

StateSpace = {"1", "2", "3", "4", "5"}


and my transition matrix is

transitionmatrix = {{17/25, 3/25, 5/25, 0, 0}, {5/18, 5/18, 7/18, 1/18, 0}, {4/21,
9/21, 4/21, 0, 4/21}, {0, 0, 3/4, 1/4, 0}, {3/4, 1/4, 0, 0, 0}}


And i know that a state "i" goes to "j" if in the transition matrix the value (i,j) is higher than zero.

I would like something like this

If[transitionmatrix {i, j} > 0, t];


And then i will have

transitionmatrix{1,2}
true


(as 1 goes to 2)

This uses Pick twice, once to select rows of the transition matrix, and again to select elements of the state space corresponding to positive elements of the matrix:

accessibleStates[i_] := Union@Flatten[
Pick[StateSpace, #, _?Positive] & /@
Pick[transitionmatrix, StateSpace, i]]

accessibleStates["4"]
(* {"3", "4"} *)


You can use patterns, e.g.

accessibleStates["1" | "4"]
(* {"1", "2", "3", "4"} *)

• Thanks exactly what i wanted :D really helpfull Jun 10, 2014 at 12:36
• but i'm not understanding what the second one does (accessibleStates["1" | "4"]) Jun 10, 2014 at 12:38
• I would guess it means states accessible from either 1 or 4. Jun 10, 2014 at 12:55
• @MarianadaCosta, the symbol | is a shorthand for Alternatives, so as Paxinum said, "1"|"4" is a pattern which matches either "1" or "4". You may have no use for this, but I thought it worth mentioning. Jun 10, 2014 at 13:37
• oh right, thanks (: Jun 10, 2014 at 14:24

As labelled fits with Mma 'natural' labeling. So,

fun[m_, i_] := Catenate@Position[Unitize@m[[i]], 1]


e.g.

Grid[Prepend[{#, fun[transitionmatrix, #]} & /@ Range[5], {"Vertex",
"Adjacent Out-vertices"}], Frame -> {True, True},
Dividers -> {All, {True, True, {False}, True}}]


Note also: DiscreteMarkovProcess:

dm = DiscreteMarkovProcess[
1, {{17/25, 3/25, 5/25, 0, 0}, {5/18, 5/18, 7/18, 1/18, 0}, {4/21,
9/21, 4/21, 0, 4/21}, {0, 0, 3/4, 1/4, 0}, {3/4, 1/4, 0, 0, 0}}]
g=Graph[dm]
MarkovProcessProperties[dm]