Statistical & Financial Consulting by Stanford PhD

Home Page

Gibbs Sampling is a way of simulating a random vector with a complex distribution if we have a simple conditional distribution of each coordinate given the other coordinates. The conditional distributions may be easier to sample than the resulting joint distribution because that is how the variables are defined. Suppose (X_{1}, ... , X_{p}) is the random vector that we have to simulate. The Gibbs sampler follows the algorithm:

1] Initialize values of the random vector: (X_{1}^{(0)}, ... , X_{p}^{(0)}).

2] Let (X_{1}^{(k)}, ... , X_{p}^{(k)}) be the current value of the random vector (X_{1}, ... , X_{p}). Simulate the new realization X_{1}^{(k+1)} of variable X_{1} out of the conditional distribution f( X_{1} | X_{2} = X_{2}^{(k)}, ... , X_{p} = X_{p}^{(k)}).

3] Now (X_{1}^{(k+1)}, X_{2}^{(k)}, ... , X_{p}^{(k)}) is the current value of the random vector (X_{1}, ... , X_{p}). Simulate the new realization X_{2}^{(k+1)} of variable X_{2} out of the conditional distribution f( X_{2} | X_{1} = X_{1}^{(k+1)}, X_{3} = X_{3}^{(k)}, ... , X_{p} = X_{p}^{(k)}).

4] Repeat step 3 for variables X_{3}, ... , X_{p} until you obtain the realization (X_{1}^{(k+1)}, ... , X_{p}^{(k+1)}).

5] Repeat steps 2 - 4 multiple times until the simulated distribution of vector (X_{1}, ... , X_{p}) converges to its true distribution.

**GIBBS SAMPLER REFERENCES
**

Robert, C. P., & Casella, G. (2004). Monte Carlo Statistical Methods (2nd ed). New York: Springer.

Casella, G., & George, E. I. (1992). Explaining the Gibbs sampler. The American Statistician 46 (3): pp. 167–174.

Liu, J. S. (2008). Monte Carlo Strategies in Scientific Computing. Springer.

Brooks, S., Gelman, A., Jones, G., & Meng, X. (2011). Handbook of Markov Chain Monte Carlo. Chapman & Hall / CRC.

Gilks, W. R., Richardson, S., & Spiegelhalter, D. (ed) (1995). Markov Chain Monte Carlo in Practice. Chapman & Hall / CRC.

Berg, B. A. (2004). Markov Chain Monte Carlo Simulations and Their Statistical Analysis (With Web-Based Fortran Code). Hackensack, NJ: World Scientific.

Lemieux, C. (2009). Monte Carlo and Quasi-Monte Carlo Sampling. Springer.

Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.

- Software and Numerical Tools for Monte Carlo Practitioner
- Monte Carlo Forum, "Numerical Recipies" Official Site

- Detailed description of the services offered in the areas of statistical consulting and financial consulting: home page, types of service, experience, case studies and payment options
- Directory of financial topics