An Application of Reliability Estimation in Longitudinal Designs Through Modeling Item-Specific Error Variance

Abstract

The purpose of the present study was to apply the methodology developed by Raykov on modeling item-specific variance for the measurement of internal consistency reliability with longitudinal data. Participants were a randomly selected sample of 500 individuals who took on a professional qualifications test in Saudi Arabia over four different occasions. Data were analyzed by use of confirmatory factor analysis, and item error variance was corrected for item specificity. The estimation of reliability involved composite index omega. Results indicated that the initially low and unacceptable levels of internal consistency reliability approached acceptable levels after accounting for item-specific variance. Findings were verified by testing whether the difference estimates of internal consistency reliability deviated from a zero-mean distribution using 10,000 replicated samples assuming a known (symmetric) or unknown (asymmetric) population distribution of the difference reliability coefficients. Percentage improvement reliability estimates indices were also estimated along with their 95% confidence intervals. Two appendices provide annotated Mplus syntax files for future use.