The Connection Between Varying Treatment Effects and the Crisis of Unreplicable Research

Abstract

We connect the replication crisis in social science to the default model of constant effects coupled with the flawed statistical approach of null hypothesis significance testing and the related problems arising from the default model of constant treatment effects. We argue that Bayesian modeling of interactions could lead to a general improvement in the communication and understanding of research results. Moving to Bayesian methods (or, more generally, multilevel approaches that incorporate external information) offers an opportunity for introspection into how social scientists interact with social phenomena to produce knowledge.