Common errors in data analysis: the apparent error rate of classification rules

Abstract

SynopsisClassification and diagnosis are concepts of fundamental importance in medicine. Yet all too frequently in published papers the only measure of performance of a classification rule is the optimistic apparent error rate. This is defined, some real examples are given illustrating how poor it is as an estimate of true future performance, and alternative measures are suggested.