Questions tagged [markov-chains]
Questions that deal with modeling and simulating random processes which switch between states under the Markov assumptions.
68
questions
2
votes
0
answers
37
views
0
votes
0
answers
17
views
How does MarkovProcessProperties[] calculate the “LimitTransitionMatrix” of a stochastic matrix?
How does Mathematica the function MarkovProcessProperties[] calculate the “LimitTransitionMatrix” of a stochastic matrix using Cesaro limit of a transition matrix? What is it fundamentally doing to ...
5
votes
2
answers
90
views
Which model behind SequencePredictorFunction?
Is the SequencePredictorFunction based on Hidden Markov Model?
If so, how does the model estimate the number of hidden states?
15
votes
2
answers
600
views
Diffusion probabilistic model in deep generative modeling
Diffusion Models (http://proceedings.mlr.press/v37/sohl-dickstein15.html and https://arxiv.org/abs/2006.11239) are generative models gaining popularity in the community of deep generative modeling, ...
- 765
3
votes
1
answer
140
views
Error in Metropolis-Hastings with three states?
I wanted to create a simple example of the Metropolis-Hastings algorithm as a sanity check but the check failed and the results do not line up. I have a vector of energies $\vec E=(E_1,E_2,E_3)$ and ...
0
votes
1
answer
87
views
Stationary Distribution of Markov Chain [closed]
I'm trying to compute the stationary distribution of the following Markov chain:
$$P=\left(
\begin{array}{cc}
p_{11} & p_{12} \\
p_{21} & p_{22} \\
\end{array}
\right)$$
I know you have to ...
- 415
0
votes
1
answer
120
views
DiscreteMarkovProcess: preparing the transition matrix and unexpected negative results
I would like to implement a Markov process.
It consists on a walk on a one-dimensional lattice, with nodes spaced $\epsilon$ from $-2$ to $2$.
The transition probabilities for $x \neq y$ are as ...
- 745
0
votes
0
answers
57
views
Unexpected PDF in `FirstPassageTimeDistribution`
I am new to Markov Processes, and while learning the discrete Markov chain with the following matrix $$P=\begin{bmatrix}\frac{1}{3} & \frac{2}{3} & 0 & 0 \\ \frac{1}{2} & \frac{1}{2} &...
- 101
3
votes
1
answer
150
views
Graphing Markov chains / decision trees
I'm looking to graph a simple one-way Markov chain, which is effectively a decision tree with transitions probabilities. One way I've got this working in here in an MWE, here's a simple Markov chain ...
- 867
15
votes
1
answer
1k
views
Metropolis-Hastings Algorithm - Significantly slower than Python
I would like to code a Metropolis Hasting algorithm and thought I would get the general scheme of things going with a toy example.
I think I got it running but my Mathematica code runs significantly ...
- 745
1
vote
2
answers
110
views
How to add the weights to the transition graph of a Markov chain?
The following working program uses
Graph and Markov Chain
...
- 1,314
4
votes
2
answers
236
views
Compute the stationary distribution of a large transition matrix
I am doing some simulations for simple random walks on directed random graphs. From a graph of n vertices, I get a n by ...
- 1,291
6
votes
2
answers
370
views
Generate an ε-machine graph from transition probability matrices
I am trying to generate this type of graphic:
see Fig.1a from https://arxiv.org/abs/0905.3587.
This process is associated with two transition matrices
$$ T^{(0)} = \begin{pmatrix}
0&p&0\\
1&...
- 539
1
vote
2
answers
305
views
How to convert a weighted, directed graph into a discrete Markov transition matrix
Suppose that I have matrix matT1 at time t and matT2 at time t+1:
...
- 4,823
2
votes
1
answer
64
views
Mathematica demonstration for Markov convergence tracking effect on input distributions in 2D (or 3D)
Is there a mathematica demo that demonstrates the convergence of Markov chains in 2 dimensions, and possibly 3, as follows?
In 2 dimensions, start with the interval determined by $x + y = 1$ for ...
- 1,430
0
votes
1
answer
58
views
Expression with Boole
Let us have an expression for example, this:
2 (-1 + p) (3 - 4 p + 2 p^2) Boole[\[FormalX] == 1] - 9 (1 - 2 p + p^2) Boole[\[FormalX] == 2] ...
For the whole ...
- 23
4
votes
3
answers
470
views
Slow computation of recursive sequences
I want to investigate the asymptotic behavior of the following recursive system:
...
- 235
0
votes
1
answer
33
views
Problem with solving a system of equations based on a matrix
I am trying to solve a system of equations based on a matrix in order to find the "stationary" matrix. the code I've written should break the matrix down into a system of equations such that some ...
- 782
4
votes
0
answers
143
views
4
votes
1
answer
302
views
The probability of reaching the absorbing states from a particular transient state?
Can I use the data available from MarkovProcessProperties to compute the probability of reaching each of the absorbing states from a particular transient state?
In an earlier post, kglr showed a ...
- 2,615
0
votes
1
answer
318
views
Finding Steady States that Meet Additional Constraints
I've written a function that returns the steady state given a regular, column-stochastic matrix. I want to use it to solve a larger problem, finding steady states that satisfy certain conditions.
The ...
- 101
0
votes
1
answer
122
views
How to simulate a Markov chain from the output of two other Markov chains?
I have a Markov chain defined by the output of two other Markov chains as time passes. (i.e., Z = X + Y).
How do I program such a Markov chain in Mathematica?
5
votes
1
answer
129
views
Plotting absorbing state probabilities from state 1
I have the following transition matrix:
...
- 2,615
22
votes
3
answers
2k
views
Efficient way to simulate thousands of Markov chains
I am currently trying to simulate relaxation of a protein population while maintaining the stochastic properties of the system. For this, I used a Markov chain to describe the temporal evolution of ...
- 1,022
8
votes
4
answers
283
views
Constructing higher order transition probability matrix
Recently I asked a question here about how to construct a transition probability matrix given the following list:
...
- 2,716
6
votes
3
answers
471
views
7
votes
2
answers
562
views
How to make a cow smaller (in BubbleChart3D plot)
I've been using random walk Metropolis to sample from the (unnormalised distribution) which is defined as $p(r)\propto e^{-100 r}$, where $r$ is the perpendicular distance of a point in 3D space from ...
- 3,147
3
votes
1
answer
425
views
Metropolis-Hastings Algorithm Problem
I've been trying to implement the Metropolis-Hastings algorithm for a while but there seems to be something weird going on.
This algorithm does not need as much of the statistics to understand, and ...
- 31
5
votes
1
answer
182
views
FindShortestPath in Markov Transition Matrix
So I have a part of transition matrix like this.
...
- 117
1
vote
1
answer
369
views
Deriving 68% confidence level for each parameter after MCMC
I used the MCMC package https://github.com/joshburkart/mathematica-mcmc to find the best parameters for model of Massive Gravity. Now I need to find the confidence 68% for each of those parameters and ...
- 621
4
votes
1
answer
227
views
MCMC problem for Acoustic Scale Cosmology
I'm using this MCMC Mathematica package https://github.com/joshburkart/mathematica-mcmc to do a MCMC on number of different data sets for a problem. I'm almost done but I have one problem. Whenever I ...
- 621
0
votes
1
answer
85
views
Evaluated To Non-Numerical Values Error using MCMC package Mathematica Via NIntegrate
I'm using this package https://github.com/joshburkart/mathematica-mcmc and am trying to evaluate the following:
...
- 621
1
vote
0
answers
125
views
Error using MCMC package Mathematica Via NIntegrate
I'm using this package https://github.com/joshburkart/mathematica-mcmc and am trying to evaluate the following:
...
- 621
1
vote
0
answers
43
views
5
votes
0
answers
764
views
Estimate process parameters of geometric Brownian motion with a two-state Markov chain
I have the following sequence.
Consider a model that follows a geometric Brownian motion whose drift switches between two different regimes representing the up trend and down trend. The regime ...
- 831
4
votes
0
answers
210
views
Why is this non-vectorized NDSolve faster than vectorized?
I'm trying to get a feeling for stochastic population models. A simple starting place is a birth-death process whose deterministic limit is the logistic equation. The idea is to model the probability <...
- 18.9k
2
votes
1
answer
115
views
Issue with EdgeLabels
I want to reproduce Shannon's telegraph Markov chain from A Mathematical Theory of Communication.
EdgeLabels isn't doing the trick here:
...
- 2,353
1
vote
0
answers
91
views
Generalization of DiscreteMarkovProcess?
I am trying to create a Markov chain using the initial state of a probabilistic cellular automaton, that is a list of length 8, as the first argument of the function ...
- 353
7
votes
0
answers
80
views
1
vote
0
answers
277
views
Training Hidden Markov Models using Mathematica
I have created a Hidden Markov Process with an initial probability vector, emission and transmission matrices.
...
- 572
1
vote
0
answers
404
views
Hidden Markov Model: emissions probabilities dependent on observable parameter
I need to fit an HMM where the emission probabilities (ep) are discrete and dependent on a known variable quantity. E.g.:
Imagine a daily time series of binary emissions ("1" or "2"). I suppose an ...
- 409
1
vote
1
answer
254
views
Nonlinear Markov chain (numerical simulation)
Suppose you have a linear Markov process, and you can write it as
x(t+1) = Ax(t).
Here x is the vector of values, and A is the transition matrix. Since this is linear, it can be solved analytically, ...
- 11
4
votes
1
answer
137
views
Use (network) graph as vertex
I have a Markov chain with graphs as the states. Basically how one network structure would evolve to another in the next period. To avoid confusion, I'll use the term "network" instead of "graph".
I ...
- 183
1
vote
0
answers
358
views
What's the algorithm behind markov classifier in Classify function for text classification?
https://www.wolfram.com/mathematica/new-in-10/highly-automated-machine-learning/find-which-author-wrote-a-text.html
The above url show the use of Classify ...
- 1,582
-1
votes
2
answers
172
views
Using Mathematica to calculate expected time to absorption [closed]
I am trying to solve a standard ETA on a birth-death process with $n$ states $\in \{0,\cdots,n-1\}$ where state $n-1$ is absorbing. Also $\mu_i$ is the expected time to absorption starting at state $i$...
- 107
2
votes
1
answer
251
views
Part specification error with InterpolatingFunction when generating a Markov Modulated Poisson Process
I want to create a Markov Modulated Poisson Process, where the arrival rates created by the states of a ContinuousMarkovProcess are fed into an ...
0
votes
2
answers
180
views
Question RE DiscreteMarkovProcess Graph Labels
I'm trying to label the arc's of my Markov chain with symbols like $\alpha$, $\beta$, etc., to show how the Markov chain looks abstractly.
So, I did this...
...
- 415
2
votes
0
answers
101
views
What do the default shapes/colors in DiscreteMarkovProcess Graph mean?
In making markov chains, I know how to do that, but I am curious as to what the different colors, and shapes of the vertexes in the graph output mean.
as example;
...
1
vote
0
answers
120
views
Fit a custom process to a data: inhomogeneous 2-state Markov chain
I would like to fit a custom process - a time inhomogeneous 2-state Markov chain, to data. The time inhomogeneity is a result of the transition probabilities varying sinusoidally through time with a ...
- 3,147
4
votes
2
answers
702
views
Time inhomogeneous Markov Chain in Mathematica
I would like to create a discrete 2-state Markov process, where the switching probabilities in the transition matrix vary with time.
I can currently do the following, which creates a process with ...
- 3,147