Meta-analysis of Comparative Therapy Outcome Research: a Critical Appraisal

Abstract

Meta-analysis is a systematic and quantitative approach to reviewing empirical literature. Its claimed benefits of superior objectivity and dependability arise from its adoption of the methodological principles and methods of primary empirical research. This paper offers a critical appraisal of these claims, with special reference to the application of meta-analysis to the therapy outcome literature. Features examined include the numerical combination of results obtained in independent studies; search procedures and inclusion criteria; and the coding of objective and qualitative features of the source studies, which are then used to study the correlates of outcome via disaggregation and multiple regression analysis. The major criticisms reviewed include allegations of overgeneralization, indiscriminate inclusion of low quality data, and idiosyncratic and unacceptable conclusions. It is concluded that the continued application and refinement of meta-analysis in the field of therapy outcomes has an important contribution to make, although not to the exclusion of other methods of integrating data. It is predicted that methods and principles currently associated with meta-analysis will increasingly find a place in all reviews of extensive tracts of empirical literature.