Considerations in the Interpretation of Cosmological Anomalies

Abstract

AbstractAnomalies drive scientific discovery – they are associated with the cutting edge of the research frontier, and thus typically exploit data in the low signal-to-noise regime. In astronomy, the prevalence of systematics –- both “known unknowns” and “unknown unknowns” – combined with increasingly large datasets, the widespread use of ad hoc estimators for anomaly detection, and the “look-elsewhere” effect, can lead to spurious false detections. In this informal note, I argue that anomaly detection leading to discoveries of new physics requires a combination of physical understanding, careful experimental design to avoid confirmation bias, and self-consistent statistical methods. These points are illustrated with several concrete examples from cosmology.