Gaussian approximation of general non-parametric posterior distributions

Abstract

AbstractIn a general class of Bayesian non-parametric models, we prove that the posterior distribution can be asymptotically approximated by a Gaussian process (GP). Our results apply to non-parametric exponential family that contains both Gaussian and non-Gaussian regression and also hold for both efficient (root-$n$) and inefficient (non-root-$n$) estimations. Our general approximation theorem does not rely on posterior conjugacy and can be verified in a class of GP priors that has a smoothing spline interpretation. In particular, the limiting posterior measure becomes prior free under a Bayesian version of ‘under-smoothing’ condition. Finally, we apply our approximation theorem to examine the asymptotic frequentist properties of Bayesian procedures such as credible regions and credible intervals.