Interrater Agreement for Final Infarct MRI Lesion Delineation

Abstract

Background and Purpose— Lesion volume measured on follow-up magnetic resonance imaging (MRI) is commonly used as an outcome parameter in clinical stroke trials. However, few studies have evaluated the optimal sequence choice and the interrater reliability of this outcome measure. The objective of this study was to quantify the geometric interrater agreement for lesion delineation of chronic infarcts on T2-weighted and fluid-attenuated inverse recovery (FLAIR) MRI. Methods— In a retrospective study of 14 patients, lesions on 90-day follow-up FLAIR and T2 fast spin echo MRI were outlined by 9 independent, blinded, experienced neuroradiologists. Voxel-wise interrater agreement was measured as (1) the volume of the intersection of individual rater’s lesion outlines relative to the mean lesion volume (overlap ratio) and (2) the Hausdorff distance between the lesion markings. Results— Mean patient age was 64.4 years (range, 45 to 79). Lesion volumes on FLAIR were, on average, 2.5 mL greater than were T2 volumes (median; P <0.001). We found considerable differences between raters’ lesion markings, but interrater agreement was consistently better on FLAIR than on T2 images, as measured by a greater overlap ratio ( P <0.0001) and a smaller Hausdorff distance ( P <0.0001) on FLAIR than on T2. Conclusions— FLAIR should be used to quantify follow-up infarct size to minimize interrater variability. Our study suggests that imaging analysis performed by 1 or a few trained readers may be preferred. Future studies should address objective and preferably automated criteria for final lesion delineation.