Comparison of Ability Metrics Obtained under Two Latent Trait Theory Procedures

Abstract

Both the BICAL and LOGIST computer programs implement a maximum likelihood procedure for jointly estimating the item and ability parameters. The two programs differ, however, with respect to (1) the an choring procedures used to overcome the metric inde terminancy of the paradigm; (2) the item characteristic curve models employed; and (3) how the examinees are grouped within the estimation process. Three sim ulated sets of item response data based upon a known underlying ability metric were used to investigate the metric recovery capabilities of the two computer pro grams. The results showed that both programs re covered a transformation of the underlying metric via a common equation, but the elements used in this equation were program specific. The transformation of the metric yielded by BICAL to the underlying metric depended only upon the item characteristic curve pa rameters, whereas the LOGIST transformation also de pended upon the frequency distribution of the esti mated ability scores over the underlying ability metric. The empirical results indicate that both transformations are quite sensitive to errors in the average value of the obtained item discrimination indices. Because LOG IST groups examinees by ability levels and BICAL does so by raw score levels, the variability of the transformed ability estimates yielded by BICAL were smaller than those from LOGIST. The results suggest that when comparing results yielded by the two com puter programs, particular attention should be paid to the characteristics of the obtained metrics.