Affiliation:
1. School of Computer Science and Technology, East China Normal University, Shanghai 200062, China
Abstract
The Hamiltonian Monte Carlo (HMC) sampling algorithm exploits Hamiltonian dynamics to construct efficient Markov Chain Monte Carlo (MCMC), which has become increasingly popular in machine learning and statistics. Since HMC uses the gradient information of the target distribution, it can explore the state space much more efficiently than random-walk proposals, but may suffer from high autocorrelation. In this paper, we propose Langevin Hamiltonian Monte Carlo (LHMC) to reduce the autocorrelation of the samples. Probabilistic inference involving multi-modal distributions is very difficult for dynamics-based MCMC samplers, which is easily trapped in the mode far away from other modes. To tackle this issue, we further propose a variational hybrid Monte Carlo (VHMC) which uses a variational distribution to explore the phase space and find new modes, and it is capable of sampling from multi-modal distributions effectively. A formal proof is provided that shows that the proposed method can converge to target distributions. Both synthetic and real datasets are used to evaluate its properties and performance. The experimental results verify the theory and show superior performance in multi-modal sampling.
Funder
Shanghai Municipal Project under Grant
National Natural Science Foundation of China
Subject
General Physics and Astronomy
Reference38 articles.
1. Sun, S., and Zhao, J. (2020). Pattern Recognition and Machine Learning, Tsinghua University Press.
2. Hybrid Monte Carlo;Duane;Phys. Lett. B,1987
3. Brooks, S., Gelman, A., Jones, G., and Meng, X. (2011). Handbook of Markov chain Monte Carlo, CRC Press.
4. Slice sampling;Neal;Ann. Stat.,2003
5. On the flexibility of the design of multiple try Metropolis schemes;Martino;Comput. Stat.,2013
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献