MOCCA-College: Preliminary Validity Evidence of a Cognitive Diagnostic Reading Comprehension Assessment

Abstract

As access to higher education increases, it is important to monitor students with special needs to facilitate the provision of appropriate resources and support. Although metrics such as the “reading readiness” ACT (formerly American College Testing) of provide insight into how many students may need such resources, they do not specify why a student may need support or how to provide that support. Increasingly, students are bringing reading comprehension struggles to college. Multiple-choice Online Causal Comprehension Assessment-College (MOCCA-College) is a new diagnostic reading comprehension assessment designed to identify who is a poor comprehender and also diagnose why they are a poor comprehender. Using reliability coefficients, receiver–operating characteristic curve analysis, and correlations, this study reports findings from the first year of a 3-year study to validate the assessment with 988 postsecondary students who took MOCCA-College, a subset of whom also provided data on other reading assessments (i.e., ACT, n = 377; Scholastic Aptitude Test [SAT], n = 192; and Nelson–Denny Reading Test [NDRT], n = 78). Despite some limitations (e.g., the sample is predominantly females from 4-year institutions), results indicate that MOCCA-College has good internal reliability, and scores are correlated with other reading assessments. Through a series of analyses of variance (ANOVAs), we also report how students identified by MOCCA-College as good and poor comprehenders differ in terms of demographics, cognitive processes used while reading, overall comprehension ability, and scores on admissions tests. Findings are discussed in terms of using MOCCA-College to help gauge which students may be at risk of reading comprehension difficulties, identify why they may be struggling, and inform directions in actionable instructional changes based on comprehension processing data.