Policy Implications of Using Significance Tests in Evaluation Research

Abstract

Significance tests have certain normative implications that are not commonly recognized and present serious complications for evaluation research. One of the conclusions reached in this article is that evaluators often make normative decisions, albeit unintentionally, when they use the traditional .05 significance level in studies with small numbers of cases. Another conclusion, however, is that evaluators cannot abandon significance tests, but should use and interpret them differently than in conventional social science research.