Comparing the Classification of Subjects by Two Independent Judges

Abstract

In a typical classification problem each subject in a sample of N subjects is allocated to one or other of k exhaustive and mutually exclusive categories. For example, a sample of families may be classified into social-class groupings in accordance with the Registrar General's classification based on the occupation of the father or father substitute. Or a sample of psychiatric patients may be classified into one or other of the diagnostic categories recommended by the W.H.O. Now it occasionally happens that the same sample of subjects is classified, for a given set of categories, independently by two different agents. For example in the Plowden Reports, Appendix 3 Table 29, a sample of children is classified into types of secondary school, Grammar, Comprehensive, Technical, etc., on the one hand according to the parent's ambition for their children and on the other hand according to the type of school in which the children were eventually placed. In situations such as the latter a k k classification table evolves in which, when the categories are arranged in the same order, the matches between the two separate classifications appear in the cells of the main diagonal of the table and the mismatches appear in the off-diagonal cells (see Table I below). The question then arises as to how to compare the two separate classifications and to measure in quantitative terms the degree of agreement between them.