Distance-Based Statistical Inference

Abstract

Statistical distances, divergences, and similar quantities have an extensive history and play an important role in the statistical and related scientific literature. This role shows up in estimation, where we often use estimators based on minimizing a distance. Distances also play a prominent role in hypothesis testing and in model selection. We review the statistical properties of distances that are often used in scientific work, present their properties, and show how they compare to each other. We discuss an approximation framework for model-based inference using statistical distances. Emphasis is placed on identifying in what sense and which statistical distances can be interpreted as loss functions and used for model assessment. We review a special class of distances, the class of quadratic distances, connect it with the classical goodness-of-fit paradigm, and demonstrate its use in the problem of assessing model fit. These methods can be used in analyzing very large samples.