Decision Reliability and Validity: Contributions and Limitations of Alternative Assessment Strategies

Abstract

The need to develop alternative models of psycho-educational assessment, more attuned to the logical and empirical requirements of intervention design, has been a central theme of school psychology and special education reform. In response to these challenges, alternative assessment and service delivery models have been proposed in which classification and assessment strategies are linked more directly to principles of intervention design. Yet, the degree to which alternative assessment models may share a number of problems with traditional diagnostic assessment practices has not been analyzed sufficiently. In this paper, the concepts of decision reliability and validity, extensions of reliability and validity theory that encompass decision outcomes, are used to frame a general analysis of three alternative assessment strategies: multiple gating, template matching, and time-series methods. We argue that relatively little knowledge may be gained through the analysis of individual scales and assessment technologies without placing them within a decision-making context and studying individual outcomes.