Three-way ROCs

Abstract

Receiver operating characteristic (ROC) analysis traditionally has dealt with dichoto mous diagnostic tasks (e.g., determining whether a disorder is present or absent). Often, however, medical problems involve distinguishing among more than two diag nostic alternatives. This article extends ROC concepts to diagnostic enterprises with three possible outcomes. For a trichotomous decision task, one can plot a ROC surface on three-dimensional coordinates; the volume under the ROC surface (VUS) equals the probability that test values will allow a decision maker to correctly sort a trio of items containing a randomly-selected member from each of three populations. Thus, the VUS summarizes global diagnostic accuracy for trichotomous tests, just as the area under a ROC curve does for a two-alternative diagnostic task. Information gain at points on the surface can be calculated just as is done for two-dimensional ROC curves, and investigators can thus compare three-way ROCs by comparing maximum information gain on each ROC surface. Key words: receiver operating characteristic, ROC, three-way ROC, three-alternative decision task, resampling, bootstrap, infor mation. (Med Decis Making 1999;19:78-89)