Prior information, sampling distributions, and the curse of dimensionality

Abstract

This tutorial addresses geometrical issues that concern the specification of high‐dimensional sampling distributions in Bayesian inversion. We illustrate that simple, low‐dimensional geometrical concepts that are sometimes used to construct such distributions may become completely distorted (and even untrue) in higher dimensional problems. This has important implications for Bayesian inversion: if a convenient sampling distribution is constructed using low dimensional geometrical concepts which cause it to differ from the distribution representing our prior information, these differences can become extremely expensive to correct in higher dimensions. Indeed, they may make a nonlinear inversion computationally intractable when this need not be the case. A crucial factor in Bayesian inversion is, therefore, whether one firmly believes in a particular prior distribution. If so, this distribution may constitute the most efficient sampling distribution, even in cases where it is not straight‐forward to draw samples from that prior distribution. The sampling artifacts described above then become irrelevant since they represent true prior beliefs.