The Quest for α: Developments in Multiple Comparison Procedures in the Quarter Century Since

Abstract

In a highly regarded work, Games (1971) presented state-of-the-art multiple comparison procedures (MCPs) for a variety of research scenarios and sought to bring order to the seemingly chaotic array of MCPs being used at that time. The current article is a sequel of sorts, placing Games’s insights in the context of many of the major developments in simultaneous and sequential inference since his article’s publication. Specifically, we address the common MCP scenarios of orthogonal contrasts, nonorthogonal contrasts, comparisons against a reference group, all possible pairwise comparisons, and exploratory post hoc contrasts, all under the assumed conditions of independent scores, normality, and homogeneity of variance. In addition, discussions of the philosophical issues surrounding the control of Type I error rates are presented.