Number Needing Review: A Novel Metric to Assess Triage Efficiency of Large Vessel Occlusion Detection Systems

Abstract

Background Endovascular thrombectomy is the gold‐standard treatment for large vessel occlusions (LVOs). A novel metric is introduced: the number needing review (NNR) to assess the triage efficiency of LVO detection systems. Methods Patients with suspected ischemic stroke and images processed by RapidAI LVO detection software over 6 months were reviewed. Only patients with LVOs of the M1 segment were included. The NNR was calculated for an M1 occlusion. Results Of 559 patients, M1 occlusion was detected in 42 patients (7.5%). RapidAI LVO had a sensitivity of 71%, specificity of 94%, positive predictive value of 49%, and negative predictive value of 92% for M1 occlusion. When gaze deviation and hyperdense sign were combined with RapidAI LVO, the specificity and positive predictive value increased to 100% for an M1 occlusion. A negative RapidAI LVO result combined with a low (<15 mL, T max >6 seconds) or high (<50 mL, T max >6 seconds) T max threshold was found to have a specificity and positive predictive value of 100% for no occlusion. The combination of gaze deviation, hyperdense sign, positive RapidAI LVO, and negative RapidAI LVO with low T max threshold yielded an NNR of 24 per 100 cases. When combined with a negative RapidAI LVO and a high T max threshold, the NNR was 16 per 100 cases. Adding National Institutes of Health Stroke Scale score <7 decreased the NNR to 9 per 100 cases. Conclusion Adding gaze deviation and hyperdense sign to the RapidAI LVO increases the specificity and positive predictive value for an M1 occlusion. When combined with a negative RapidAI LVO detection and either a low or high T max >6 seconds threshold, the NNR is significantly reduced.