Validation of acute myocardial infarction (AMI) in electronic medical records: the SPEED-EXTRACT Study

Abstract

AbstractObjectivesTo determine whether data captured in electronic medical records (eMR) is sufficient to serve as a clinical data source to make a reliable determination of ST elevation myocardial infarction (STEMI) and non-ST elevation myocardial infarction (NSTEMI) and to use these eMR derived diagnoses to validate ICD-10 codes for STEMI and NSTEMI.DesignRetrospective validation by blind chart review of a purposive sample of patients with a troponin test result, ECG record, and medical note available in the eMR.SettingTwo local health districts containing two tertiary hospitals and six referral hospitals in New South Wales, Australia.ParticipantsN = 897 adult patients who had a hs-troponin test result indicating suspected AMI.Primary outcome measuresInter-rater reliability of clinical diagnosis (κ) for ST-elevated myocardial infarction (STEMI) and Non-ST elevated myocardial infarction (NSTEMI); and sensitivity, specificity, and positive predictive value (PPV) of ICD-10 codes for STEMI and NSTEMI.ResultsThe diagnostic agreement between clinical experts was high for STEMI (κ = 0.786) but lower for NSTEMI (κ = 0.548). ICD-10 STEMI codes had moderate sensitivity (Se = 88±6.7), very high specificity (Sp = 99±0.7) and high positive predictive value (PPV = 91±6). NSTEMI ICD-10 codes were lower in each case (Se = 69±6.4, Sp = 96.0±1.5, PPV = 84±6).ConclusionsThe eMR held sufficient clinical data to reliably diagnose STEMI, producing high inter-rater agreement among our expert reviewers as well as allowing reasonably precise estimates of the accuracy of administrative ICD-10 codes. However the clinical detail held in the eMR was less sufficient to diagnose NSTEMI, indicated by a lower inter-rater agreement. Efforts should be directed towards operationalising the clinical definition of NSTEMI and improving clinical record keeping to enable an accurate description of the clinical phenotype in the eMR, and thus improve reliability of the diagnosis of NSTEMI using these data sources.Article SummaryStrengths and limitations of this studyExpert chart review provided a robust evaluation of the reliability and sufficiency of data directly extracted from the EMR for the diagnosis of AMIComputational interrogation and extraction of the eMR (via SPEED-EXTRACT) allowed us to use a wide selection for inclusion in the sample on the basis of clinical data independent of ICD-10 code, enabling the capture of missed cases (i.e., uncoded AMI) and so determine estimates for the false negative rate and sensitivityResults were necessarily based on the subset of patients with sufficient clinical data in the eMR. Inferences from this subset to the wider patient pool will be biased when the availability of records varies with diagnosisAt least two sources of uncertainty in the gold reference standard we used are indistinguishable: uncertainty due to poor clinical detail in the eMR, and uncertainty due to a weak operational definition of the diagnosis (e.g., NSTEMI).