Artificial intelligence stenosis diagnosis in coronary CTA: effect on the performance and consistency of readers with less cardiovascular experience

Abstract

Abstract Background To investigate the influence of artificial intelligence (AI) based on deep learning on the diagnostic performance and consistency of inexperienced cardiovascular radiologists. Methods We enrolled 196 patents who had undergone both coronary computed tomography angiography (CCTA) and invasive coronary angiography (ICA) within 6 months. Four readers with less cardiovascular experience (Reader 1–Reader 4) and two cardiovascular radiologists (level II, Reader 5 and Reader 6) evaluated all images for ≥ 50% coronary artery stenosis, with ICA as the gold standard. Reader 3 and Reader 4 interpreted with AI system assistance, and the other four readers interpreted without the AI system. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy (area under the receiver operating characteristic curve (AUC)) of the six readers were calculated at the patient and vessel levels. Additionally, we evaluated the interobserver consistency between Reader 1 and Reader 2, Reader 3 and Reader 4, and Reader 5 and Reader 6. Results The AI system had 94% and 78% sensitivity at the patient and vessel levels, respectively, which were higher than that of Reader 5 and Reader 6. AI-assisted Reader 3 and Reader 4 had higher sensitivity (range + 7.2–+ 16.6% and + 5.9–+ 16.1%, respectively) and NPVs (range + 3.7–+ 13.4% and + 2.7–+ 4.2%, respectively) than Reader 1 and Reader 2 without AI. Good interobserver consistency was found between Reader 3 and Reader 4 in interpreting ≥ 50% stenosis (Kappa value = 0.75 and 0.80 at the patient and vessel levels, respectively). Only Reader 1 and Reader 2 showed poor interobserver consistency (Kappa value = 0.25 and 0.37). Reader 5 and Reader 6 showed moderate agreement (Kappa value = 0.55 and 0.61). Conclusions Our study showed that using AI could effectively increase the sensitivity of inexperienced readers and significantly improve the consistency of coronary stenosis diagnosis via CCTA. Trial registration Clinical trial registration number: ChiCTR1900021867. Name of registry: Diagnostic performance of artificial intelligence-assisted coronary computed tomography angiography for the assessment of coronary atherosclerotic stenosis.