Inference for matched tuples and fully blocked factorial designs

Abstract

This paper studies inference in randomized controlled trials with multiple treatments, where treatment status is determined according to a “matched tuples” design. If there are | D | possible treatments, then by a matched tuples design, we mean an experimental design where units are sampled i.i.d. from the population of interest, grouped into “homogeneous” blocks of size | D | , and finally, within each block, exactly one individual is randomly assigned to each of the | D | treatments. We first study estimation and inference for matched tuples designs in the general setting where the parameter of interest is a vector of linear contrasts over the collection of average potential outcomes for each treatment. Parameters of this form include standard average treatment effects used to compare one treatment relative to another, but also include parameters that may be of interest in the analysis of factorial designs. We first establish conditions under which a sample analog estimator is asymptotically normal and construct a consistent estimator of its corresponding asymptotic variance. Combining these results establish the asymptotic exactness of tests based on these estimators. In contrast, we show that, for two common testing procedures based on t‐tests constructed from linear regressions, one test is generally conservative while the other is generally invalid. We go on to apply our results to study the asymptotic properties of what we call “fully‐blocked” 2 K factorial designs, which are simply matched tuples designs applied to a full factorial experiment. Leveraging our previous results, we establish that our estimator achieves a lower asymptotic variance under the fully‐blocked design than that under any stratified factorial design, which stratifies the experimental sample into a finite number of “large” strata. A simulation study and empirical application illustrate the practical relevance of our results.