Gains in Accuracy from Replicated Readings of Diagnostic Images

Abstract

From the perspective of signal detection theory, human variation in image reading degrades diagnostic accuracy by broadening the statistical distributions of perceived evidence upon which decisions were based. A new multivariate "random-effects" model formulates the total variation in a diagnostic decision variable as a sum of three uncorrelated components that represent differences among cases, readers, and repeated readings by a given reader. This model provides a basis for quantitative predictions concerning the amount by which diagnostic accuracy, as specified by ROC analysis, can be enhanced by the replication of image readings. Although these predictions apply exactly only to the hypothetical situation in which normally distributed decision variables from equivalent readers are averaged, computer- simulation studies and an analysis of mammographic image-reading data from five radiol ogists show that similar gains in accuracy can be achieved by averaging discrete confidence ratings. Key words diagnostic accuracy, random-effects model; image interpretation; ROC analysis. (Med Decis Making 1992;12:60-75)