Comparison of screening methods for computer adaptive tests to predict reading and math performance

Abstract

AbstractThe present study compared the diagnostic accuracy of a single computer adaptive test (CAT), Star Reading or Star Math, and a combination of the two in a gated screening framework to predict end‐of‐year proficiency in reading and math. Participants included 13,009 students in Grades 3–8 who had at least one fall screening score and end‐of‐year state test score in reading and math. First, diagnostic accuracy statistics were evaluated for a single screening measure to predict proficiency on end‐of‐year tests. Second, a gated screening framework was simulated to examine the diagnostic accuracy of a combination of screening measures (i.e., scores from the CATs and the end‐of‐year test). The diagnostic accuracy of each screening method was compared. Results suggest that diagnostic accuracy did not improve for the gated screening method when compared to the single screening method. The gated screening method tended to yield low sensitivity values (M = 0.42, range = 0.35–0.48) and high specificity values (M = 0.97, range = 0.95–0.99). The only condition to reach acceptable sensitivity and specificity (>0.70) was a single reading screener predicting reading outcomes. Sample specific cut‐scores from receiver operating curve (ROC) analyses led to improved diagnostic accuracy outcomes relative to all other methods.