Development of an initial training and evaluation programme for manual lower limb muscle MRI segmentation

Abstract

Abstract Background Magnetic resonance imaging (MRI) quantification of intramuscular fat accumulation is a responsive biomarker in neuromuscular diseases. Despite emergence of automated methods, manual muscle segmentation remains an essential foundation. We aimed to develop a training programme for new observers to demonstrate competence in lower limb muscle segmentation and establish reliability benchmarks for future human observers and machine learning segmentation packages. Methods The learning phase of the training programme comprised a training manual, direct instruction, and eight lower limb MRI scans with reference standard large and small regions of interest (ROIs). The assessment phase used test–retest scans from two patients and two healthy controls. Interscan and interobserver reliability metrics were calculated to identify underperforming outliers and to determine competency benchmarks. Results Three experienced observers undertook the assessment phase, whilst eight new observers completed the full training programme. Two of the new observers were identified as underperforming outliers, relating to variation in size or consistency of segmentations; six had interscan and interobserver reliability equivalent to those of experienced observers. The calculated benchmark for the Sørensen-Dice similarity coefficient between observers was greater than 0.87 and 0.92 for individual thigh and calf muscles, respectively. Interscan and interobserver reliability were significantly higher for large than small ROIs (all p < 0.001). Conclusions We developed, implemented, and analysed the first formal training programme for manual lower limb muscle segmentation. Large ROI showed superior reliability to small ROI for fat fraction assessment. Relevance statement Observers competent in lower limb muscle segmentation are critical to application of quantitative muscle MRI biomarkers in neuromuscular diseases. This study has established competency benchmarks for future human observers or automated segmentation methods. Key points • Observers competent in muscle segmentation are critical for quantitative muscle MRI biomarkers. • A training programme for muscle segmentation was undertaken by eight new observers. • We established competency benchmarks for future human observers or automated segmentation methods. Graphical Abstract