Variability of Deviation IQ's Based on Multiple Choice Test Scores

Abstract

A computer program simulated guessing on multiple-choice test items and calculated deviation IQ's from observed scores which contained a guessing component. The entire procedure was replicated 5000 times for each of a variety of assumptions about the number of test items, the number of choices per item, and the distribution of the number of items "known" by examinees before guessing. Extensive variability in deviation IQ's due entirely to chance was found. The degree of variability was sensitive to the scale location of the distribution of number of items "known" but not to the shape or variance of that distribution. For all distributions, the degree of variability consistently depended inversely on the number of items and inversely on the number of choices per item.