Multi-Specialty Expert Physician Identification of Extranodal Extension in Computed Tomography Scans of Oropharyngeal Cancer Patients: Prospective Blinded Human Inter-Observer Performance Evaluation
Author:
, Sahin OnurORCID, Wahid Kareem A., Taku Nicolette, He Renjie, Naser Mohamed A., Mohamed Abdallah S. R., Mäkitie Antti, Kann Benjamin H., Kaski Kimmo, Sahlsten Jaakko, Jaskari Joel, Amit Moran, Chen Melissa M., Chronowski Gregory M., Diaz Eduardo M., Garden Adam S., Goepfert Ryan P., Guenette Jeffrey P., Gunn G. Brandon, Hirvonen Jussi, Hoebers Frank, Guha-Thakurta Nandita, Johnson Jason, Kaya Diana, Khanpara Shekhar D., Nyman Kristofer, Lai Stephen Y., Lango Miriam, Learned Kim O., Lee Anna, Lewis Carol M., Maniakas Anastasios, Moreno Amy C., Myers Jeffery N., Phan Jack, Pytynia Kristen B., Rosenthal David I., Sandulache Vlad, Schellingerhout Dawid, Shah Shalin J., Sikora Andrew G., Wintermark Max, Fuller Clifton D.ORCID
Abstract
ABSTRACTBackgroundExtranodal extension (ENE) is an important adverse prognostic factor in oropharyngeal cancer (OPC) and is often employed in therapeutic decision making. Clinician-based determination of ENE from radiological imaging is a difficult task with high inter-observer variability. However, the role of clinical specialty on the determination of ENE has been unexplored.MethodsPre-therapy computed tomography (CT) images for 24 human papillomavirus-positive (HPV+) OPC patients were selected for the analysis; 6 scans were randomly chosen to be duplicated, resulting in a total of 30 scans of which 21 had pathologically-confirmed ENE. 34 expert clinician annotators, comprised of 11 radiologists, 12 surgeons, and 11 radiation oncologists separately evaluated the 30 CT scans for ENE and noted the presence or absence of specific radiographic criteria and confidence in their prediction. Discriminative performance was measured using accuracy, sensitivity, specificity, area under the receiver operating characteristic curve (AUC), and Brier score for each physician. Statistical comparisons of discriminative performance were calculated using Mann Whitney U tests. Significant radiographic factors in correct discrimination of ENE status were determined through a logistic regression analysis. Interobserver agreement was measured using Fleiss’ kappa.ResultsThe median accuracy for ENE discrimination across all specialties was 0.57. There were significant differences between radiologists and surgeons for Brier score (0.33 vs. 0.26), radiation oncologists and surgeons for sensitivity (0.48 vs. 0.69), and radiation oncologists and radiologists/surgeons for specificity (0.89 vs. 0.56). There were no significant differences between specialties for accuracy or AUC. Indistinct capsular contour, nodal necrosis, and nodal matting were significant factors in regression analysis. Fleiss’ kappa was less than 0.6 for all the radiographic criteria, regardless of specialty.ConclusionsDetection of ENE in HPV+OPC patients on CT imaging remains a difficult task with high variability, regardless of clinician specialty. Although some differences do exist between the specialists, they are often minimal. Further research in automated analysis of ENE from radiographic images is likely needed.
Publisher
Cold Spring Harbor Laboratory
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
|
|