Emergent Principles for the Design, Implementation, and Analysis of Cluster-Based Experiments in Social Science

Abstract

In experimentally designed research, many good reasons exist for assigning groups or clusters to treatments rather than individuals. This article discusses them. But cluster-level designs face some unique or exacerbated challenges. The article identifies them and offers some principles about them. One emphasizes how statistical power and sample size estimation depend on intraclass correlations, particularly after conditioning on the use of cluster-level covariates. Another stresses assigning experimental units at the lowest level of aggregation possible, provided this does not subtly change the research question. A third emphasizes the utility of minimizing and measuring interunit communication, though neither is easy to achieve. A fourth advises against experiments that are totally black box and so leave program implementation and process unstudied, though such study often makes the research process more salient. The last principle involves the utility of describing treatment heterogeneity and estimating its consequences, though causal conclusions about the heterogeneity will be less well warranted compared to conclusions about the intended treatment, every experiment's major focus.