Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [markov-process]

The tag has no usage guidance.

4
votes
1answer
158 views

The probability of reaching the absorbing states from a particular transient state?

Can I use the data available from MarkovProcessProperties to compute the probability of reaching each of the absorbing states from a particular transient state? In an earlier post, kglr showed a ...
1
vote
0answers
34 views

Initial values in EstimatedProcess with Gaussian HiddenMarkovProcess

I am using the following code to get states and transitions out of some traces: class = HiddenMarkovProcess[2, "Gaussian"]; hmm = EstimatedProcess[traces, class]; ...
4
votes
1answer
64 views

Plotting absorbing state probabilities from state 1

I have the following transition matrix: ...
5
votes
4answers
177 views

Constructing higher order transition probability matrix

Recently I asked a question here about how to construct a transition probability matrix given the following list: ...
4
votes
3answers
243 views

Constructing transition probability matrix

I have the following list: ...
1
vote
0answers
51 views

How to simulate two coupled birth-death-immigration processes?

I am simulating a birth-death-immigration process for two coupled populations that interact by virtue of the birth-rate in one population being equal to the death-rate in the other population. The two ...
1
vote
0answers
52 views

Speed up for SequencePredictorFunction?

From Google Dive: SequencePredictorFunction is here, along with my test data. From pastbin: prediction function and data is here. Using the ...
11
votes
1answer
2k views

Markov Random Field

I look for an efficient way to implement a Pairwise Markov Random Field. I have implemented a homogeneous (i.e., unary potential is a similar function in all nodes, and pairwise potential is a ...
5
votes
0answers
492 views

Estimate process parameters of geometric Brownian motion with a two-state Markov chain

I have the following sequence. Consider a model that follows a geometric Brownian motion whose drift switches between two different regimes representing the up trend and down trend. The regime ...
2
votes
0answers
84 views

Future support for Hierarchical Hidden Markov Model in Mathematica

I know that Hidden Markov Models are supported, but Hierarchical Hidden Markov Models (that are the one used in speech recognition and most other fields where people think HMMs are used) are not yet ...
1
vote
0answers
199 views

Training Hidden Markov Models using Mathematica

I have created a Hidden Markov Process with an initial probability vector, emission and transmission matrices. ...
1
vote
1answer
208 views

Nonlinear Markov chain (numerical simulation)

Suppose you have a linear Markov process, and you can write it as x(t+1) = Ax(t). Here x is the vector of values, and A is the transition matrix. Since this is linear, it can be solved analytically, ...
3
votes
0answers
43 views

WeightedAdjacencyMatrix does not complete for Connected Graph

Please see code snippet below - I have a weighted, connected graph that models a software process using a DiscreteMarkovProcess. I set some of the edge weights to ...
1
vote
1answer
333 views

How to generate a graph from a tetrahedral lattice

My goal is to generate a graph from a 3D isotropic array of vertex connected tetrahedrons. Each tetrahedron represents a node of my graph, and connects to 4 other neighbors. Then my idea is to ...
11
votes
1answer
678 views

How to implement QLearning?

I know that v10 has support for both Machine Learning and Finite Markov Processes, the Q-learning algorithm uses both. Specifically, Q-learning finds an optimal action-selection policy for any given (...