The Predicament of Large Numbers of Observations and How We Got There: Critical Review

Abstract

Abstract Background Classical statistics were developed in a time when small sample sizes were the norm; thus, statistical significance typically ensured large clinical effects. Over the past 10–20 years, computational techniques have allowed studies with modest effects to reach statistical significance (usually P < 0.05) by analyzing very large numbers of patients. In this review, I discuss how this came about and provide an intuitive understanding of the strengths and weaknesses of various statistical parameters that provide insight into clinical effect sizes. Content In this review of the literature, a simple web-based program was used for calculations. Examples are shown. Odds and risk ratios are compared with ROC curves to allow better understanding of their predictive value. Summary In these complex times, an intuitive understanding of statistical procedures is increasingly important. This review will attempt to advance the reader’s knowledge so that one can calculate the number needed to treat and its confidence interval, understand the meaning of a modest association, and determine when a study is likely to be accurate but with questionable clinical utility.