# Questions tagged [markov-process]

The tag has no usage guidance.

15 questions
1answer
158 views

### The probability of reaching the absorbing states from a particular transient state?

Can I use the data available from MarkovProcessProperties to compute the probability of reaching each of the absorbing states from a particular transient state? In an earlier post, kglr showed a ...
0answers
34 views

### Initial values in EstimatedProcess with Gaussian HiddenMarkovProcess

I am using the following code to get states and transitions out of some traces: class = HiddenMarkovProcess[2, "Gaussian"]; hmm = EstimatedProcess[traces, class]; ...
1answer
64 views

### Plotting absorbing state probabilities from state 1

I have the following transition matrix: ...
4answers
177 views

### Constructing higher order transition probability matrix

Recently I asked a question here about how to construct a transition probability matrix given the following list: ...
3answers
243 views

### Constructing transition probability matrix

I have the following list: ...
0answers
51 views

### How to simulate two coupled birth-death-immigration processes?

I am simulating a birth-death-immigration process for two coupled populations that interact by virtue of the birth-rate in one population being equal to the death-rate in the other population. The two ...
0answers
52 views

### Speed up for SequencePredictorFunction?

From Google Dive: SequencePredictorFunction is here, along with my test data. From pastbin: prediction function and data is here. Using the ...
1answer
2k views

### Markov Random Field

I look for an efficient way to implement a Pairwise Markov Random Field. I have implemented a homogeneous (i.e., unary potential is a similar function in all nodes, and pairwise potential is a ...
0answers
492 views

### Estimate process parameters of geometric Brownian motion with a two-state Markov chain

I have the following sequence. Consider a model that follows a geometric Brownian motion whose drift switches between two different regimes representing the up trend and down trend. The regime ...
0answers
84 views

### Future support for Hierarchical Hidden Markov Model in Mathematica

I know that Hidden Markov Models are supported, but Hierarchical Hidden Markov Models (that are the one used in speech recognition and most other fields where people think HMMs are used) are not yet ...
0answers
199 views

### Training Hidden Markov Models using Mathematica

I have created a Hidden Markov Process with an initial probability vector, emission and transmission matrices. ...
1answer
208 views

### Nonlinear Markov chain (numerical simulation)

Suppose you have a linear Markov process, and you can write it as x(t+1) = Ax(t). Here x is the vector of values, and A is the transition matrix. Since this is linear, it can be solved analytically, ...
0answers
43 views

### WeightedAdjacencyMatrix does not complete for Connected Graph

Please see code snippet below - I have a weighted, connected graph that models a software process using a DiscreteMarkovProcess. I set some of the edge weights to ...
1answer
333 views

### How to generate a graph from a tetrahedral lattice

My goal is to generate a graph from a 3D isotropic array of vertex connected tetrahedrons. Each tetrahedron represents a node of my graph, and connects to 4 other neighbors. Then my idea is to ...
1answer
678 views

### How to implement QLearning?

I know that v10 has support for both Machine Learning and Finite Markov Processes, the Q-learning algorithm uses both. Specifically, Q-learning finds an optimal action-selection policy for any given (...