Some Thoughts about the Suitability of the Reliable Change Index (RCI) for Analysis of Ordinal Scale Data

Abstract

The reliable change index (RCI) was introduced approximately 30 decades ago in order to provide an empirical, statistically grounded technique for determining whether improvement after a therapeutic intervention was real or due to measurement error. Since the definitions of the properties and limitations of scales of measurement described by Stevens in 1947, there has been vigorous controversy about whether it is permissible to analyse ordinal data with parametric statistics. Specifically, are parameters and statistics such as means and standard deviations meaningful in the context of ordinal data? These are important concerns because many of the scales used to measure outcomes in behavioural research and clinical settings yield ordinal-scale measures. Given that the standard deviation is used in the computation of the RCI, the question as to whether or not the RCI is reliable when used with ordinal-scale data is explored. Data from the SPRS-2 was used to calculate minimum reliable difference criteria in terms of both (ordinal) Total Raw Scores (MRDRS) and logit scores (MRDLS) derived from Rasch analysis. Test–retest differences across the Total Raw Score range were evaluated using each criterion. At both extremes of the range, small changes in Total Raw Score not deemed to be reliable according to the MRDRScriterion were classified as reliable according to the MRDLScriterion. Conversely, test–retest changes in the centre of the range deemed to be reliable according to the MRDRScriterion were classified as unreliable according to the MRDLScriterion. It is suggested that while MRDRScan determinenumericallyreliable differences, MRDLScan determine reliable differences that are meaningful in terms of the underlying construct being measured.