Multitrait-Multimethod Analysis of FCAT Reading and Writing

Abstract

This study investigated the convergent and discriminant validity of the high-stakes Florida Comprehensive Assessment Test (FCAT) in both reading and writing at grade levels 4, 8, and 10. The data from the 2006 FCAT administration were analyzed via traditional multitrait-multimethod (MTMM) analysis to identify the factor structure and structural equation models (SEMs), to determine the weights of the influential variables underlying the tests. The MTMM analyses suggested that across all grade levels, the correlation between the multiple-choice reading and multiple-choice writing tests of the FCAT approached the reliability coefficients for each test separately. These correlations were higher than the correlations between the multiple choice and performance sections for each trait. The SEM analyses, however, provided support for both the convergent and discriminant validity of the test scores. The fit for the SEMs improved from a CFI of .84 for 4th graders (a poor fit) to .97 for 10th-grade students.