Author:
Byrd Catherine,Ajawara Ureka,Laundry Ryan,Radin John,Bhandari Prasha,Leung Ann,Han Summer,Asch Stephen M.,Zeliadt Steven,Harris Alex H. S.,Backhus Leah
Abstract
Abstract
Background
We aim to develop and test performance of a semi-automated method (computerized query combined with manual review) for chart abstraction in the identification and characterization of surveillance radiology imaging for post-treatment non-small cell lung cancer patients.
Methods
A gold standard dataset consisting of 3011 radiology reports from 361 lung cancer patients treated at the Veterans Health Administration from 2008 to 2016 was manually created by an abstractor coding image type, image indication, and image findings. Computerized queries using a text search tool were performed to code reports. The primary endpoint of query performance was evaluated by sensitivity, positive predictive value (PPV), and F1 score. The secondary endpoint of efficiency compared semi-automated abstraction time to manual abstraction time using a separate dataset and the Wilcoxon rank-sum test.
Results
Query for image type demonstrated the highest sensitivity of 85%, PPV 95%, and F1 score 0.90. Query for image indication demonstrated sensitivity 72%, PPV 70%, and F1 score 0.71. The image findings queries ranged from sensitivity 75–85%, PPV 23–25%, and F1 score 0.36–0.37. Semi-automated abstraction with our best performing query (image type) improved abstraction times by 68% per patient compared to manual abstraction alone (from median 21.5 min (interquartile range 16.0) to 6.9 min (interquartile range 9.5), p < 0.005).
Conclusions
Semi-automated abstraction using the best performing query of image type improved abstraction efficiency while preserving data accuracy. The computerized query acts as a pre-processing tool for manual abstraction by restricting effort to relevant images. Determining image indication and findings requires the addition of manual review for a semi-automatic abstraction approach in order to ensure data accuracy.
Funder
Health Services Research and Development
Publisher
Springer Science and Business Media LLC
Subject
Health Informatics,Health Policy,Computer Science Applications
Reference32 articles.
1. American Cancer Society-Cancer Facts & Figures 2020. Atlanta: American Cancer Society; 2020.
2. Henschke CI, Yankelevitz DF, Libby DM, Pasmantier MW, et al. Survival of patients with stage I lung cancer detected on CT screening. N Engl J Med. 2006;355(17):1763–71.
3. Johnson BE, Cortazar P, Chute JP. Second lung cancers in patients successfully treated for lung cancer [abstract]. Semin Oncol. 1997;24(4):492–9.
4. Vanmeerbeeck J. Second primary lung cancer in Flanders: frequency, clinical presentation, treatment and prognosis. Lung Cancer. 1996;15(3):281–95.
5. Carrell DS, Halgrim S, Tran D-T, Buist DSM, Chubak J, Chapman WW, et al. Using natural language processing to improve efficiency of manual chart abstraction in research: the case of breast cancer recurrence. Am J Epidemiol. 2014;179(6):749–58.