Abstract
The Bernstein-von Mises theorem, also known as the Bayesian Central Limit Theorem (BCLT), states that under certain assumptions a posterior distribution can be approximated as a multivariate normal distribution as long as the number of pseudo-observations is large. We derive a form of the BCLT for the canonical conjugate prior of a regular exponential family distribution using the machinery of information geometry. Our approach applies the core approximation for the BCLT, Laplace's method, to the free-entropy (i.e., log-normalizer) of an exponential family distribution. Additionally, we formulate approximations for the Kullback-Leibler divergence and Fisher-Rao metric on the conjugate prior manifold in terms of corresponding quantities from the likelihood manifold. We also include an application to the categorical distribution and show that the free-entropy derived approximations are related to various series expansions of the gamma function and its derivatives. Furthermore, for the categorical distribution, the free-entropy approximation produces higher order expansions than the BCLT alone.