COT 4401 Top 10 Algorithms Chris Lacher Markov Chain Monte Carlo / Metropolis Algorithm

Resources

• Video 1: Markov Chain Monte Carlo Simulation
• Video 2: MCMC Example: Metropolis
• History 1: Turing's Cathedral: The Origins of the Digital Universe. George Dyson (author). Pantheon, 2012. ISBN-13: 978-0375422775 ISBN-10: 0375422773.
  Authored by the son a famed physist Freeman Dyson who grew up at the Institute for Advanced Study. This is a remarkable account of the early days of computing during and after WWII. Meet Turing, von Neuman, Ulam, and many other pioneers as they build the first computers, devise algorithms, and help win the war.
  Review
• History 2: Stan Ulam, John von Neumann, and the Monte Carlo Method
  A historical perspective by Roger Eckhardt, written as a Los Alamos white paper. [Los Alamos Science Special Issue, 1987]
• History 3: Original von Neuman letter dated Apr 9, 1947 (complete), courtesy of Michael Mascagni
  
• Paper 1: Wikipedia on the Original Metropolis et al Paper
• Paper 2: Travelling Salesman
  Chapter from a physics course at Harvey Mudd College Describes use of MCMC/Metropolis to find (approximate) solutions to Travelling Salesman Problem. [Peter Saeta, HMC, Physics 170]
• Paper 3 [advanced]: Neural Network Learning
  A sample application of MCMC/Metropolis in learning algorithms for artificial neural networks ["Reversible Jump MCMC Simulated Annealing for Neural Networks", Uncertainty in Artificial Intelligence Conference Proceedings 2000, pp 11-18].
• Paper 4 [advanced]: Bayesian Protein Structure Prediction
  A sample application of MCMC/Metropolis in predicting protein structural folding [Duke University Department of Statistics].

Simple Example: Find the volume of a complicated region

Given a bounded region R in space, we can estimate the size of R as follows: First, find a cube that contains the region, lets assume the cube is the unit cube I³, where I is the closed interval [0, 1]. Then choose "random" points p₁, p₂, ... , p_k from I³ and test whether each point lies in R. Then the number of points found to be in R divided by k is an estimate of the probability that a random point in the cube belongs to R. Looked at another way, this is an estimate of the volume of R.

To see why even a "volume" calculation may be difficult to do without Monte Carlo, consider that the region may be the graph of a function of two variables, then the volume of region would be the integral of the function over the square I². Monte Carlo gives us a way to estimate the value of this double integral when a closed form for the integral is unknown.

In another light, note that the dimension of our region and cube might be much larger than 3, say 250. While in dimension 3 we could replace Monte Carlo with an exhaustive estimate of the volume by testing every point in the hypercube whose coordinates are multiples of 0.01, thus testing over a grid with mesh 0.01. But in the 250 dimensional hypercube, there are 100²⁵⁰ = 10²⁵⁰⁰ points in our "sample". This number is too large to deal with in today's computational world. By employing Monte Carlo, the volume estimate is tractable.

There are of course issues: How accurate is the estimate, and how confident should we be in that accuracy? Study of these questions has consumed much energy and produced a great deal of interesting and useful statistical theory, including Markov Chain Monte Carlo.

Exercises

• What is the project that Ulam et al were working on when they were inventing Monte Carlo methods? Briefly describe the project, and describe why it was important to use Monte Carlo. What were they needing to know?
• Describe the Markov Chain Monte Carlo framework.
• Describe the diversity of modern applications of MCMC / Metropolis.