Skip to main content

Questions tagged [markov-chains]

Questions that deal with modeling and simulating random processes which switch between states under the Markov assumptions.

Filter by
Sorted by
Tagged with
1 vote
1 answer
97 views

How to find periodic steady state of symbolic matrix in Mathematica?

I am currently trying to recreate results in a paper where I solve for the periodic steady state of the following rate matrix: Where epsilon is some force parameter, I have found the left and right ...
magg13__'s user avatar
0 votes
1 answer
57 views

How to solve a Markov transition matrix using Mathematica?

I need some help in working out how to solve this problem. I have a transition matrix of size $4\times4$ at time $t=0$ - call it $A$ I also have another transition matrix at time $t= 10$ - call it $B$ ...
ifty's user avatar
  • 9
0 votes
1 answer
84 views

Any luck estimating a multivariate hidden Markov process?

I am having problems using EstimatedProcess to estimate a multivariate HiddenMarkovProcess. I'm following the work of a paper, so I know this method is feasible. In brief, given 2 economic variables: ...
XXX's user avatar
  • 1
2 votes
2 answers
107 views

Optimization of a Markov сhain with symbolic transition rate

I am trying to work on 1D random walk that can move to left, right or stay with probabilities $p_i$,$q_i$,$r_i$ that changes with the site $i$. I am trying to simulate this by using a recurrence ...
Hugo Andrade's user avatar
2 votes
0 answers
55 views

Discrete Markov Process generator

...
Alkis Tsiatsios's user avatar
5 votes
2 answers
97 views

Which model behind SequencePredictorFunction?

Is the SequencePredictorFunction based on Hidden Markov Model? If so, how does the model estimate the number of hidden states?
andrea simonetti's user avatar
17 votes
2 answers
1k views

Diffusion probabilistic model in deep generative modeling

Diffusion Models (http://proceedings.mlr.press/v37/sohl-dickstein15.html and https://arxiv.org/abs/2006.11239) are generative models gaining popularity in the community of deep generative modeling, ...
Baoxiang Pan's user avatar
3 votes
1 answer
157 views

Error in Metropolis-Hastings with three states?

I wanted to create a simple example of the Metropolis-Hastings algorithm as a sanity check but the check failed and the results do not line up. I have a vector of energies $\vec E=(E_1,E_2,E_3)$ and ...
AccidentalTaylorExpansion's user avatar
0 votes
1 answer
124 views

Stationary Distribution of Markov Chain [closed]

I'm trying to compute the stationary distribution of the following Markov chain: $$P=\left( \begin{array}{cc} p_{11} & p_{12} \\ p_{21} & p_{22} \\ \end{array} \right)$$ I know you have to ...
PiE's user avatar
  • 437
0 votes
1 answer
135 views

DiscreteMarkovProcess: preparing the transition matrix and unexpected negative results

I would like to implement a Markov process. It consists on a walk on a one-dimensional lattice, with nodes spaced $\epsilon$ from $-2$ to $2$. The transition probabilities for $x \neq y$ are as ...
Smerdjakov's user avatar
0 votes
0 answers
57 views

Unexpected PDF in `FirstPassageTimeDistribution`

I am new to Markov Processes, and while learning the discrete Markov chain with the following matrix $$P=\begin{bmatrix}\frac{1}{3} & \frac{2}{3} & 0 & 0 \\ \frac{1}{2} & \frac{1}{2} &...
Jeremy's user avatar
  • 101
3 votes
1 answer
277 views

Graphing Markov chains / decision trees

I'm looking to graph a simple one-way Markov chain, which is effectively a decision tree with transitions probabilities. One way I've got this working in here in an MWE, here's a simple Markov chain ...
DRG's user avatar
  • 877
15 votes
1 answer
1k views

Metropolis-Hastings Algorithm - Significantly slower than Python

I would like to code a Metropolis Hasting algorithm and thought I would get the general scheme of things going with a toy example. I think I got it running but my Mathematica code runs significantly ...
Smerdjakov's user avatar
1 vote
2 answers
135 views

How to add the weights to the transition graph of a Markov chain?

The following working program uses Graph and Markov Chain ...
florin's user avatar
  • 1,798
4 votes
2 answers
337 views

Compute the stationary distribution of a large transition matrix

I am doing some simulations for simple random walks on directed random graphs. From a graph of n vertices, I get a n by ...
LeafGlowPath's user avatar
  • 1,453
6 votes
2 answers
399 views

Generate an ε-machine graph from transition probability matrices

I am trying to generate this type of graphic: see Fig.1a from https://arxiv.org/abs/0905.3587. This process is associated with two transition matrices $$ T^{(0)} = \begin{pmatrix} 0&p&0\\ 1&...
AG1123's user avatar
  • 571
1 vote
2 answers
403 views

How to convert a weighted, directed graph into a discrete Markov transition matrix

Suppose that I have matrix matT1 at time t and matT2 at time t+1: ...
Tugrul Temel's user avatar
  • 6,223
2 votes
1 answer
74 views

Mathematica demonstration for Markov convergence tracking effect on input distributions in 2D (or 3D)

Is there a mathematica demo that demonstrates the convergence of Markov chains in 2 dimensions, and possibly 3, as follows? In 2 dimensions, start with the interval determined by $x + y = 1$ for ...
ExpressionCoder's user avatar
0 votes
1 answer
60 views

Expression with Boole

Let us have an expression for example, this: 2 (-1 + p) (3 - 4 p + 2 p^2) Boole[\[FormalX] == 1] - 9 (1 - 2 p + p^2) Boole[\[FormalX] == 2] ... For the whole ...
hardcheese's user avatar
4 votes
3 answers
479 views

Slow computation of recursive sequences

I want to investigate the asymptotic behavior of the following recursive system: ...
David Lingard's user avatar
0 votes
1 answer
37 views

Problem with solving a system of equations based on a matrix

I am trying to solve a system of equations based on a matrix in order to find the "stationary" matrix. the code I've written should break the matrix down into a system of equations such that some ...
Wombles's user avatar
  • 792
4 votes
0 answers
147 views

Fit Markov Chain

Assume a simple Markov Chain with the transition matrix ...
tukan's user avatar
  • 407
4 votes
1 answer
343 views

The probability of reaching the absorbing states from a particular transient state?

Can I use the data available from MarkovProcessProperties to compute the probability of reaching each of the absorbing states from a particular transient state? In an earlier post, kglr showed a ...
user120911's user avatar
  • 2,685
0 votes
1 answer
332 views

Finding Steady States that Meet Additional Constraints

I've written a function that returns the steady state given a regular, column-stochastic matrix. I want to use it to solve a larger problem, finding steady states that satisfy certain conditions. The ...
Jonathan's user avatar
  • 101
1 vote
1 answer
135 views

How to simulate a Markov chain from the output of two other Markov chains?

I have a Markov chain defined by the output of two other Markov chains as time passes. (i.e., Z = X + Y). How do I program such a Markov chain in Mathematica?
MathematicaIsAwesome's user avatar
5 votes
1 answer
135 views

Plotting absorbing state probabilities from state 1

I have the following transition matrix: ...
user120911's user avatar
  • 2,685
22 votes
3 answers
2k views

Efficient way to simulate thousands of Markov chains

I am currently trying to simulate relaxation of a protein population while maintaining the stochastic properties of the system. For this, I used a Markov chain to describe the temporal evolution of ...
BPinto's user avatar
  • 1,022
8 votes
4 answers
321 views

Constructing higher order transition probability matrix

Recently I asked a question here about how to construct a transition probability matrix given the following list: ...
Wiliam's user avatar
  • 3,026
6 votes
3 answers
583 views

Constructing transition probability matrix

I have the following list: ...
Wiliam's user avatar
  • 3,026
7 votes
2 answers
659 views

How to make a cow smaller (in BubbleChart3D plot)

I've been using random walk Metropolis to sample from the (unnormalised distribution) which is defined as $p(r)\propto e^{-100 r}$, where $r$ is the perpendicular distance of a point in 3D space from ...
ben18785's user avatar
  • 3,167
3 votes
1 answer
489 views

Metropolis-Hastings Algorithm Problem

I've been trying to implement the Metropolis-Hastings algorithm for a while but there seems to be something weird going on. This algorithm does not need as much of the statistics to understand, and ...
MathDude's user avatar
6 votes
1 answer
190 views

FindShortestPath in Markov Transition Matrix

So I have a part of transition matrix like this. ...
sdc's user avatar
  • 127
1 vote
1 answer
382 views

Deriving 68% confidence level for each parameter after MCMC

I used the MCMC package https://github.com/joshburkart/mathematica-mcmc to find the best parameters for model of Massive Gravity. Now I need to find the confidence 68% for each of those parameters and ...
Daniel Berkowitz's user avatar
4 votes
1 answer
235 views

MCMC problem for Acoustic Scale Cosmology

I'm using this MCMC Mathematica package https://github.com/joshburkart/mathematica-mcmc to do a MCMC on number of different data sets for a problem. I'm almost done but I have one problem. Whenever I ...
Daniel Berkowitz's user avatar
0 votes
1 answer
86 views

Evaluated To Non-Numerical Values Error using MCMC package Mathematica Via NIntegrate

I'm using this package https://github.com/joshburkart/mathematica-mcmc and am trying to evaluate the following: ...
Daniel Berkowitz's user avatar
1 vote
0 answers
134 views

Error using MCMC package Mathematica Via NIntegrate

I'm using this package https://github.com/joshburkart/mathematica-mcmc and am trying to evaluate the following: ...
Daniel Berkowitz's user avatar
1 vote
0 answers
44 views

What's going on with this FirstPassageTimeDistribution performance quirk?

Observe: ...
ciao's user avatar
  • 25.8k
5 votes
0 answers
786 views

Estimate process parameters of geometric Brownian motion with a two-state Markov chain

I have the following sequence. Consider a model that follows a geometric Brownian motion whose drift switches between two different regimes representing the up trend and down trend. The regime ...
user19218's user avatar
  • 841
4 votes
0 answers
212 views

Why is this non-vectorized NDSolve faster than vectorized?

I'm trying to get a feeling for stochastic population models. A simple starting place is a birth-death process whose deterministic limit is the logistic equation. The idea is to model the probability <...
Chris K's user avatar
  • 20.3k
2 votes
1 answer
128 views

Issue with EdgeLabels

I want to reproduce Shannon's telegraph Markov chain from A Mathematical Theory of Communication. EdgeLabels isn't doing the trick here: ...
andandandand's user avatar
  • 2,393
1 vote
0 answers
95 views

Generalization of DiscreteMarkovProcess?

I am trying to create a Markov chain using the initial state of a probabilistic cellular automaton, that is a list of length 8, as the first argument of the function ...
Lame-Ov2.0's user avatar
7 votes
0 answers
80 views

Problem/bug with `RandomVariate` and `FirstPassageTimeDistribution`?

...
ciao's user avatar
  • 25.8k
1 vote
0 answers
287 views

Training Hidden Markov Models using Mathematica

I have created a Hidden Markov Process with an initial probability vector, emission and transmission matrices. ...
Vahagn Tumanyan's user avatar
1 vote
0 answers
412 views

Hidden Markov Model: emissions probabilities dependent on observable parameter

I need to fit an HMM where the emission probabilities (ep) are discrete and dependent on a known variable quantity. E.g.: Imagine a daily time series of binary emissions ("1" or "2"). I suppose an ...
CupiDio's user avatar
  • 439
1 vote
1 answer
275 views

Nonlinear Markov chain (numerical simulation)

Suppose you have a linear Markov process, and you can write it as x(t+1) = Ax(t). Here x is the vector of values, and A is the transition matrix. Since this is linear, it can be solved analytically, ...
samchand's user avatar
4 votes
1 answer
143 views

Use (network) graph as vertex

I have a Markov chain with graphs as the states. Basically how one network structure would evolve to another in the next period. To avoid confusion, I'll use the term "network" instead of "graph". I ...
Art's user avatar
  • 183
1 vote
0 answers
359 views

What's the algorithm behind markov classifier in Classify function for text classification?

https://www.wolfram.com/mathematica/new-in-10/highly-automated-machine-learning/find-which-author-wrote-a-text.html The above url show the use of Classify ...
m00nlight's user avatar
  • 1,582
-1 votes
2 answers
183 views

Using Mathematica to calculate expected time to absorption [closed]

I am trying to solve a standard ETA on a birth-death process with $n$ states $\in \{0,\cdots,n-1\}$ where state $n-1$ is absorbing. Also $\mu_i$ is the expected time to absorption starting at state $i$...
carefullynamed's user avatar
2 votes
1 answer
258 views

Part specification error with InterpolatingFunction when generating a Markov Modulated Poisson Process

I want to create a Markov Modulated Poisson Process, where the arrival rates created by the states of a ContinuousMarkovProcess are fed into an ...
Thomas Fankhauser's user avatar
0 votes
2 answers
182 views

Question RE DiscreteMarkovProcess Graph Labels

I'm trying to label the arc's of my Markov chain with symbols like $\alpha$, $\beta$, etc., to show how the Markov chain looks abstractly. So, I did this... ...
PiE's user avatar
  • 437