# Questions tagged [markov-process]

The tag has no usage guidance.

27 questions
Filter by
Sorted by
Tagged with
65 views

### How to solve a Markov transition matrix using Mathematica?

I need some help in working out how to solve this problem. I have a transition matrix of size $4\times4$ at time $t=0$ - call it $A$ I also have another transition matrix at time $t= 10$ - call it $B$ ...
• 9
1 vote
156 views

### Use Mathematica to implement a Penney's game solver

In Penney's game , player A selects a sequence of heads and tails (of length 3 or larger), and shows this sequence to player B. Player B then selects another sequence of heads and tails of the same ...
• 1,683
87 views

### Any luck estimating a multivariate hidden Markov process?

I am having problems using EstimatedProcess to estimate a multivariate HiddenMarkovProcess. I'm following the work of a paper, so I know this method is feasible. In brief, given 2 economic variables: ...
• 1
139 views

### Replacing 2D Patterns in 2D Lists | Markov Programming Language

Recently I came across with a probabilistic programming language where programs are combinations of rewrite rules - Markov Junior I think it is quite interesting to try out implementing some features ...
• 1,255
56 views

...
158 views

### Error in Metropolis-Hastings with three states?

I wanted to create a simple example of the Metropolis-Hastings algorithm as a sanity check but the check failed and the results do not line up. I have a vector of energies $\vec E=(E_1,E_2,E_3)$ and ...
136 views

### DiscreteMarkovProcess: preparing the transition matrix and unexpected negative results

I would like to implement a Markov process. It consists on a walk on a one-dimensional lattice, with nodes spaced $\epsilon$ from $-2$ to $2$. The transition probabilities for $x \neq y$ are as ...
• 775
57 views

• 571
1 vote
40 views

### Given W(t) as a standard Wiener process, how do I plot $t \, W(1)$ for $0\leq t \leq 1$?

I've plotted $W(t)$ using timestep = 1/250 data = RandomFunction[WienerProcess[0, 1], {0, 1, timestep}] ListLinePlot[%, Filling -> Axis] Now I need to plot \$W(...
• 47
36 views

### Fix parameter in EstimateProcess

Can I fix some parameter in EstimateProcess? I want to estimate the transition matrix of a hidden markov process without optimization/change of the distribution ...
• 41
343 views

### The probability of reaching the absorbing states from a particular transient state?

Can I use the data available from MarkovProcessProperties to compute the probability of reaching each of the absorbing states from a particular transient state? In an earlier post, kglr showed a ...
• 2,695
1 vote
48 views

### Initial values in EstimatedProcess with Gaussian HiddenMarkovProcess

I am using the following code to get states and transitions out of some traces: class = HiddenMarkovProcess[2, "Gaussian"]; hmm = EstimatedProcess[traces, class]; ...
• 41
135 views

### Plotting absorbing state probabilities from state 1

I have the following transition matrix: ...
• 2,695
327 views

### Constructing higher order transition probability matrix

Recently I asked a question here about how to construct a transition probability matrix given the following list: ...
• 3,106
587 views

### Constructing transition probability matrix

I have the following list: ...
• 3,106
1 vote
154 views

### How to simulate two coupled birth-death-immigration processes?

I am simulating a birth-death-immigration process for two coupled populations that interact by virtue of the birth-rate in one population being equal to the death-rate in the other population. The two ...
• 115
1 vote
65 views

### Speed up for SequencePredictorFunction?

From Google Dive: SequencePredictorFunction is here, along with my test data. From pastbin: prediction function and data is here. Using the ...
• 4,996
2k views

### Markov Random Field

I look for an efficient way to implement a Pairwise Markov Random Field. I have implemented a homogeneous (i.e., unary potential is a similar function in all nodes, and pairwise potential is a ...
• 2,019
794 views

### Estimate process parameters of geometric Brownian motion with a two-state Markov chain

I have the following sequence. Consider a model that follows a geometric Brownian motion whose drift switches between two different regimes representing the up trend and down trend. The regime ...
• 841
144 views

### Future support for Hierarchical Hidden Markov Model in Mathematica

I know that Hidden Markov Models are supported, but Hierarchical Hidden Markov Models (that are the one used in speech recognition and most other fields where people think HMMs are used) are not yet ...
1 vote
291 views

### Training Hidden Markov Models using Mathematica

I have created a Hidden Markov Process with an initial probability vector, emission and transmission matrices. ...
1 vote
277 views

### Nonlinear Markov chain (numerical simulation)

Suppose you have a linear Markov process, and you can write it as x(t+1) = Ax(t). Here x is the vector of values, and A is the transition matrix. Since this is linear, it can be solved analytically, ...
• 11
45 views

### WeightedAdjacencyMatrix does not complete for Connected Graph

Please see code snippet below - I have a weighted, connected graph that models a software process using a DiscreteMarkovProcess. I set some of the edge weights to ...
1 vote
528 views

### How to generate a graph from a tetrahedral lattice

My goal is to generate a graph from a 3D isotropic array of vertex connected tetrahedrons. Each tetrahedron represents a node of my graph, and connects to 4 other neighbors. Then my idea is to ...
• 13