An Empirical Power Analysis of Quasi-Exact Tests for the Rasch Model

Abstract

Measurement invariance is not only an important requirement of tests but also a central point in the examination of the Rasch model. Ponocny (2001) suggested quasi-exact tests for small samples which allow for formulating test-statistics based on matrices obtained using Monte Carlo methods. The purpose of the present study was to analyze the type-I error rates and the empirical power of two test-statistics for the assumption of measurement invariance in comparison with Andersen’s likelihood ratio test (1973). Each simulation was based on 10,000 replications and was a function of sample size (n = 30, 50, 100, 200), test length (k = 5, 9, 17), varying number of items exhibiting model violation, magnitude of violation, and different ability distributions. The results indicate that it is possible to detect large model violations on item level with samples of n = 50 or n = 100, and even weak violations with n = 200. Additionally, the results showed that it is possible to investigate very small samples where a parametric approach is not possible, which is one of the most important advantages of quasi-exact tests.