The Strength of the Methodological Warrants for the Findings of Research on Program Evaluation Use

Abstract

Considerable research has been conducted on the use of the findings of program evaluation, but little, if any, attention has been paid to the soundness of the methods of this research. If the methods are not sound or not well described in the research, the strength of the conclusions of the research is unknown. The authors examine the empirical studies reported in five widely cited reviews of the literature on program evaluation use and summarize the types of methods used, the professions for which they were conducted, and the extent to which they met two basic criteria for soundness. It was concluded that, by and large, the body of studies does not adequately address the criteria. With care, the findings of the research literature on the use of evaluations might provide guidance for evaluators, but they lack sufficient scientific credibility for other uses.