Undependability of Construct Validity Patterns for Tests and Items1

Abstract

Validity scores for items were obtained by comparing the rank order of the correlations between each item and six criterion measures with the corresponding rank order for the test as a whole. For samples of about 85 cases, the validity pattern for each of two tests showed limited consistency from sample to sample and the item validity scores had low correlations between samples. While sample size contributed to these undependabilities, the additional undependability of item responses themselves raises strong doubts about the practicality of selecting items on the basis of their validity patterns.