The Many Threats from Mechanistic Heterogeneity That Can Spoil Multimethod Research

Abstract

AbstractThe combination of cross-case and within-case analysis in Multi-Method Research (MMR) designs has gained considerable traction in the social sciences over the last decade. One reason for the popularity of MMR is grounded in the idea that different methods can complement each other, in the sense that the strengths of one method can compensate for the blind spots and weaknesses of another and vice versa. In this chapter, we critically address this core premise of MMR with an emphasis on the external validity of applying some cross-case method, like standard regression or Qualitative Comparative Analysis, in combination with case study analysis. After a brief overview of the rationale of MMR, we discuss in detail the problem of deriving generalizable claims about mechanisms in research contexts that likely exhibit mechanistic heterogeneity. In doing so, we clarify what we mean by mechanistic heterogeneity and where researchers should look for potential sources of mechanistic heterogeneity. Finally, we propose a strategy for progressively updating our confidence in the external validity of claims about causal mechanisms through the strategic selection of cases for within-case analysis based on the diversity of the population.