Abstract
Abstract
Objective
To determine whether an artificial intelligence candidate could pass the rapid (radiographic) reporting component of the Fellowship of the Royal College of Radiologists (FRCR) examination.
Design
Prospective multi-reader diagnostic accuracy study.
Setting
United Kingdom.
Participants
One artificial intelligence candidate (Smarturgences, Milvue) and 26 radiologists who had passed the FRCR examination in the preceding 12 months.
Main outcome measures
Accuracy and pass rate of the artificial intelligence compared with radiologists across 10 mock FRCR rapid reporting examinations (each examination containing 30 radiographs, requiring 90% accuracy rate to pass).
Results
When non-interpretable images were excluded from the analysis, the artificial intelligence candidate achieved an average overall accuracy of 79.5% (95% confidence interval 74.1% to 84.3%) and passed two of 10 mock FRCR examinations. The average radiologist achieved an average accuracy of 84.8% (76.1-91.9%) and passed four of 10 mock examinations. The sensitivity for the artificial intelligence was 83.6% (95% confidence interval 76.2% to 89.4%) and the specificity was 75.2% (66.7% to 82.5%), compared with summary estimates across all radiologists of 84.1% (81.0% to 87.0%) and 87.3% (85.0% to 89.3%). Across 148/300 radiographs that were correctly interpreted by >90% of radiologists, the artificial intelligence candidate was incorrect in 14/148 (9%). In 20/300 radiographs that most (>50%) radiologists interpreted incorrectly, the artificial intelligence candidate was correct in 10/20 (50%). Most imaging pitfalls related to interpretation of musculoskeletal rather than chest radiographs.
Conclusions
When special dispensation for the artificial intelligence candidate was provided (that is, exclusion of non-interpretable images), the artificial intelligence candidate was able to pass two of 10 mock examinations. Potential exists for the artificial intelligence candidate to improve its radiographic interpretation skills by focusing on musculoskeletal cases and learning to interpret radiographs of the axial skeleton and abdomen that are currently considered “non-interpretable.”
Funder
National Institute for Health and Care Research
Reference19 articles.
1. Mukherjee S. A.I. versus M.D: What happens when diagnosis is automated? 2017. https://www.newyorker.com/magazine/2017/04/03/ai-versus-md.
2. The Fellowship of the Royal College of Radiologists (FRCR) examination: a review of the evidence
3. Radiology training in United Kingdom: current status;Kassamali;Quant Imaging Med Surg,2014
4. The Royal College of Radiologists. Final Examination for the Fellowship in Clinical Radiology (Part B) Scoring System. https://www.rcr.ac.uk/sites/default/files/docs/radiology/pdf/CR2B_scoring_system.pdf.
5. The Royal College of Radiologists. FRCR Part 2B (Radiology) - CR2B Examination Information. 2022. https://www.rcr.ac.uk/clinical-radiology/examinations/frcr-part-2b-radiology-cr2b.
Cited by
29 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献