Including Omission Mistakes in the Calculation of Cohen's Kappa and an Analysis of the Coefficient's Paradox Features

Abstract

The application range of Cohen's Kappa is extended to the field of sequential observation data, where omission mistakes of an observer may often occur. It is shown how the omission mistakes can be incorporated into the calculation of the Kappa coefficient without violating the statistic it is based on. The enhanced coefficient is termed Kappa Coefficient With Omission Calculation (K woc). In addition, the so-called paradox features of the Kappa coefficient, the base-rate problem, and the observer bias are discussed. It is shown that, in the case of observation data, these attributes are desirable features of the Kappa coefficient.