Statistical versus Economic Significance in Accounting: A Reality Check

Abstract

Abstract Empirical research is ripe for a reality check, as elegantly put by the “elephants in the room” (Ohlson, 2022a. Empirical accounting seminars: Elephants in the room. Accounting, Economics, and Law: Convivium.) referring to practices to disguise false positives with the aid of statistical engineering. However, the diagnosis points to a deeper problem. The dominant empirical paradigm combines extraordinarily vague hypotheses with ridiculously high desired levels of statistical confidence beatable solely with econometric hacks. Instead, I argue that economic magnitudes measure meaningful theoretical constructs and require far less than conventional significance levels for measurements of sufficient importance. Precisely estimating that an effect is close to zero can be more meaningful than a noisy but significant coefficient. I make several actionable proposals: (1) report standard errors rather than conventional statistical significance (stars) or t-stats, (2) discuss target significance levels likely to change priors and could much higher than weak significance for unsettled questions, (3) report precisely estimated zeros and power analyses, and (4) anchor empirical design on formal theory justified with precise references or structural models.