Bias in Meta-Analytic Estimates of the Absolute Efficacy of Psychotherapy

Abstract

Meta-analytic estimates of the absolute efficacy of psychotherapy indicate an effect size of approximately 0.80. However, various biases in primary and meta-analytic studies may have influenced this estimate. This study examines 4 nonsystematic biases that increase error variance (i.e., nonrandomized designs, methodological deficiencies, failure to use the study as the unit of analysis, and violations of homogeneity), 4 underestimation biases that primarily concern psychometric issues (i.e., unreliability of outcome measures, failure to report nonsignificant effect sizes, nonoptimal composite outcome measures, and nonstandardized outcome measures), and 8 overestimation biases (i.e., excluding nonsignificant effects from calculations of effect size estimates, failure to adjust for small sample bias, failure to separate studies using single-group pre–post designs vs. control group designs, using unweighted average effect sizes, analyzing biased partial samples that reflect treatment dropout and research attrition, researcher allegiance bias, publication bias, and wait-list control group bias). Wherever possible, evidence regarding the magnitude of these biases is presented, and methods for addressing these biases separately and collectively are discussed. Implications of the meta-analytic evidence on psychotherapy for the effect sizes of other psychological interventions are also considered.