An Empirical Link of Content and Construct Validity Evidence

Abstract

Since the 1940s, measurement specialists have called for an empirical validation technique that combines con tent- and construct-related evidence. This study investi gated the value of such a technique. A self-assessment instrument designed to cover four traditional foreign language skills was administered to 1,404 college-level foreign language students. Four subject-matter experts were asked to provide item dissimilarity judgments, us ing whatever criteria they thought appropriate. The data from the students and the experts were examined sepa rately using multidimensional scaling followed by clus ter and discriminant analyses. Results showed that the structure of the data underlying both the student and ex pert scaling solutions corresponded closely to that speci fied in the instrument blueprint. In addition, using canonical correlation, a comparison of the two scaling solutions revealed a high degree of similarity in the two solutions.