Effect of training and different measurement strategies on the reproducibility of brain MRI lesion load measurements in multiple sclerosis

Abstract

In this study, we evaluated the intra- and interobserver variabilities in measuring lesion load of brain MRI abnormalities present on proton-density scans from patients with MS, using using both manual outlining or a semiautomated local thresholding technique (LTT). We also evaluated how these variabilities were affected by the use of standard rules for lesion load measurements, training, and different measurement strategies. The intraobserver variabilities obtained after establishing rules for lesion load measurements and training were not significantly different from those obtained before any consensus among the observers, both for manual outlining and for the LTT. On the contrary, the interobserver variabilities obtained with manual outlining or the LTT were significantly reduced when rules for lesion load measurements were used. For manual outlining, the intraobserver variability did not significantly change when the measurements were performed after an experienced radiologist identified lesions or when using adjacent slices and the corresponding T2-weighted images as reference for lesion identification. On the contrary, for the LTT, the intraobserver variability was significantly reduced by the use of the radiologic marking. The interobserver variabilities for both manual outlining and the LTT were reduced compared with the free condition when these measurement strategies were used. Our findings demonstrate that both lesion identification and outlining are important sources of variation for MRI lesion load measurements in MS and that there are simple strategies to reduce such variation that might be useful when planning clinical trials.