Affiliation:
1. Department of Statistical Science, University College London, London WC1E 6BT, UK
Abstract
Developing an efficient computational scheme for high-dimensional Bayesian variable selection in generalised linear models and survival models has always been a challenging problem due to the absence of closed-form solutions to the marginal likelihood. The Reversible Jump Markov Chain Monte Carlo (RJMCMC) approach can be employed to jointly sample models and coefficients, but the effective design of the trans-dimensional jumps of RJMCMC can be challenging, making it hard to implement. Alternatively, the marginal likelihood can be derived conditional on latent variables using a data-augmentation scheme (e.g., Pólya-gamma data augmentation for logistic regression) or using other estimation methods. However, suitable data-augmentation schemes are not available for every generalised linear model and survival model, and estimating the marginal likelihood using a Laplace approximation or a correlated pseudo-marginal method can be computationally expensive. In this paper, three main contributions are presented. Firstly, we present an extended Point-wise implementation of Adaptive Random Neighbourhood Informed proposal (PARNI) to efficiently sample models directly from the marginal posterior distributions of generalised linear models and survival models. Secondly, in light of the recently proposed approximate Laplace approximation, we describe an efficient and accurate estimation method for marginal likelihood that involves adaptive parameters. Additionally, we describe a new method to adapt the algorithmic tuning parameters of the PARNI proposal by replacing Rao-Blackwellised estimates with the combination of a warm-start estimate and the ergodic average. We present numerous numerical results from simulated data and eight high-dimensional genetic mapping data-sets to showcase the efficiency of the novel PARNI proposal compared with the baseline add–delete–swap proposal.
Funder
UK Engineering and Physical Sciences Research Council
Subject
General Physics and Astronomy
Reference78 articles.
1. Hastie, T., Tibshirani, R., and Wainwright, M. (2015). Statistical Learning with Sparsity: The Lasso and Generalizations, CRC Press.
2. Akaike, H. (1998). Selected Papers of Hirotugu Akaike, Springer.
3. Estimating the dimension of a model;Schwarz;Ann. Stat.,1978
4. Bayesian measures of model complexity and fit;Spiegelhalter;J. R. Stat. Soc. Ser. B Stat. Methodol.,2002
5. Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory;Watanabe;J. Mach. Learn. Res.,2010