Skip to main content
Mathematica

Return to Answer

added 508 characters in body
Source Link
mef
mef
  • 1.6k
  • 11
  • 16

The following does not answer the OP's question directly, in that it does not provide modifications of the code presented. The purpose of this "answer" is to provide a clear statement of the Metropolis-Hastings algorithm and its relation to the Metropolis algorithm in hopes that this would aid the OP in modifying the code him- or herself. My impression from reading the OP's question is that there may be a misunderstanding on the OP's part, but perhaps I am wrong and this "answer" should be removed.

Given data (vector) $y$ and unknown parameter (vector) $\theta$, the posterior distribution is given by $$ p(\theta|y) = \frac{p(y|\theta)\,p(\theta)}{p(y)} , $$ where $p(y|\theta)$ is the likelihood, $p(\theta)$ is the prior, and $p(y)$ is (often called) the marginal likelihood.

The Metropolis-Hastings algorithm can be summarized as follows. Let $q(\theta'|\theta)$ denote the density for the proposal distribution for $\theta'$ conditioned on $\theta$. Let $\theta^{(r)}$ denote the current state of the MCMC chain. Then \begin{equation} \theta^{(r+1)} = \begin{cases} \theta' & \mathcal{R} \ge u \\ \theta^{(r)} & \text{otherwise} \end{cases} , \end{equation} where $u \sim \textsf{Uniform}(0,1)$ and \begin{equation} \mathcal{R} = \frac{p(\theta'|y)}{p(\theta^{(r)}|y)} \times \frac{q(\theta^{(r)}|\theta')}{q(\theta'|\theta^{(r)})} . \end{equation} Note that $p(y)$ cancels out of the first factor on the right-hand side. Also note that if \begin{equation} q(\theta|\theta') = q(\theta'|\theta) \qquad\text{for all $(\theta,\theta')$} \end{equation} then the second factor disappears and we have the Metropolis algorithm.

Given data (vector) $y$ and unknown parameter (vector) $\theta$, the posterior distribution is given by $$ p(\theta|y) = \frac{p(y|\theta)\,p(\theta)}{p(y)} , $$ where $p(y|\theta)$ is the likelihood, $p(\theta)$ is the prior, and $p(y)$ is (often called) the marginal likelihood.

The Metropolis-Hastings algorithm can be summarized as follows. Let $q(\theta'|\theta)$ denote the density for the proposal distribution for $\theta'$ conditioned on $\theta$. Let $\theta^{(r)}$ denote the current state of the MCMC chain. Then \begin{equation} \theta^{(r+1)} = \begin{cases} \theta' & \mathcal{R} \ge u \\ \theta^{(r)} & \text{otherwise} \end{cases} , \end{equation} where $u \sim \textsf{Uniform}(0,1)$ and \begin{equation} \mathcal{R} = \frac{p(\theta'|y)}{p(\theta^{(r)}|y)} \times \frac{q(\theta^{(r)}|\theta')}{q(\theta'|\theta^{(r)})} . \end{equation} Note that $p(y)$ cancels out of the first factor on the right-hand side. Also note that if \begin{equation} q(\theta|\theta') = q(\theta'|\theta) \qquad\text{for all $(\theta,\theta')$} \end{equation} then the second factor disappears and we have the Metropolis algorithm.

The following does not answer the OP's question directly, in that it does not provide modifications of the code presented. The purpose of this "answer" is to provide a clear statement of the Metropolis-Hastings algorithm and its relation to the Metropolis algorithm in hopes that this would aid the OP in modifying the code him- or herself. My impression from reading the OP's question is that there may be a misunderstanding on the OP's part, but perhaps I am wrong and this "answer" should be removed.

Given data (vector) $y$ and unknown parameter (vector) $\theta$, the posterior distribution is given by $$ p(\theta|y) = \frac{p(y|\theta)\,p(\theta)}{p(y)} , $$ where $p(y|\theta)$ is the likelihood, $p(\theta)$ is the prior, and $p(y)$ is (often called) the marginal likelihood.

The Metropolis-Hastings algorithm can be summarized as follows. Let $q(\theta'|\theta)$ denote the density for the proposal distribution for $\theta'$ conditioned on $\theta$. Let $\theta^{(r)}$ denote the current state of the MCMC chain. Then \begin{equation} \theta^{(r+1)} = \begin{cases} \theta' & \mathcal{R} \ge u \\ \theta^{(r)} & \text{otherwise} \end{cases} , \end{equation} where $u \sim \textsf{Uniform}(0,1)$ and \begin{equation} \mathcal{R} = \frac{p(\theta'|y)}{p(\theta^{(r)}|y)} \times \frac{q(\theta^{(r)}|\theta')}{q(\theta'|\theta^{(r)})} . \end{equation} Note that $p(y)$ cancels out of the first factor on the right-hand side. Also note that if \begin{equation} q(\theta|\theta') = q(\theta'|\theta) \qquad\text{for all $(\theta,\theta')$} \end{equation} then the second factor disappears and we have the Metropolis algorithm.

Source Link
mef
mef
  • 1.6k
  • 11
  • 16

Given data (vector) $y$ and unknown parameter (vector) $\theta$, the posterior distribution is given by $$ p(\theta|y) = \frac{p(y|\theta)\,p(\theta)}{p(y)} , $$ where $p(y|\theta)$ is the likelihood, $p(\theta)$ is the prior, and $p(y)$ is (often called) the marginal likelihood.

The Metropolis-Hastings algorithm can be summarized as follows. Let $q(\theta'|\theta)$ denote the density for the proposal distribution for $\theta'$ conditioned on $\theta$. Let $\theta^{(r)}$ denote the current state of the MCMC chain. Then \begin{equation} \theta^{(r+1)} = \begin{cases} \theta' & \mathcal{R} \ge u \\ \theta^{(r)} & \text{otherwise} \end{cases} , \end{equation} where $u \sim \textsf{Uniform}(0,1)$ and \begin{equation} \mathcal{R} = \frac{p(\theta'|y)}{p(\theta^{(r)}|y)} \times \frac{q(\theta^{(r)}|\theta')}{q(\theta'|\theta^{(r)})} . \end{equation} Note that $p(y)$ cancels out of the first factor on the right-hand side. Also note that if \begin{equation} q(\theta|\theta') = q(\theta'|\theta) \qquad\text{for all $(\theta,\theta')$} \end{equation} then the second factor disappears and we have the Metropolis algorithm.