A Method for Evaluating Computer Programs for Electrocardiographic Interpretation

Abstract

A previously described method for evaluating computer programs for electrocardiographic (ECG) interpretation was applied to Version D of the Public Health Service (PHS) program and to the Mayo Clinic program of 1968. Staff cardiologists found agreement with the results of the PHS program in 45.5% of 1150 unselected tracings. Clinically significant disagreements based strictly on application of different criteria occurred in 29%, while disagreements based on program errors were found in 25.5%. The corresponding results for the Mayo Clinic program are: agreement in 47%, disagreements due to criteria differences in 30.9%, and disagreements due to program errors in 22.1%. Both programs had serious deficiencies, particularly in the diagnostic categories of myocardial infarction and cardiac arrhythmias. PHS program errors resulted primarily from mismeasurements and deficient program logic, while Mayo Clinic program errors more frequently resulted from pattern recognition failures. Neither program appears suitable for routine clinical use at the present time.