Insertion of synthetic lesions on patient data: a method for evaluating clinical performance differences between PET systems

Abstract

Abstract Background Performance assessment of positron emission tomography (PET) scanners is crucial to guide clinical practice with efficiency. We have already introduced and experimentally evaluated a simulation method allowing the creation of a controlled ground truth for system performance assessment. In the current study, the goal was to validate the method using patient data and demonstrate its relevance to assess PET performances accuracy in clinical conditions. Methods Twenty-four patients were recruited and sorted into two groups according to their body mass index (BMI). They were administered with a single dose of 2 MBq/kg 18F-FDG and scanned using clinical protocols consecutively on two PET systems: the Discovery-IQ (DIQ) and the Discovery-MI (DMI). For each BMI group, sixty synthetic lesions were dispatched in three subgroups and inserted at relevant anatomical locations. Insertion of synthetic lesions (ISL) was performed at the same location into the two consecutive exams. Two nuclear medicine physicians evaluated individually and blindly the images by qualitatively and semi-quantitatively reporting each detected lesion and agreed on a consensus. We assessed the inter-system detection rates of synthetic lesions and compared it to an initial estimate of at least 1.7 more targets detected on the DMI and the detection rates of natural lesions. We determined the inter-reader variability, evaluated according to the inter-observer agreement (IOA). Adequate inter-reader variability was found for IOA above 80%. Differences in standardized uptake value (SUV) metrics were also studied. Results In the BMI ≤ 25 group, the relative true positive rate (RTPR) for synthetic and natural lesions was 1.79 and 1.83, respectively. In the BMI > 25 group, the RTPR for synthetic and natural lesions was 2.03 and 2.27, respectively. For each BMI group, the detection rate using ISL was consistent to our estimate and with the detection rate measured on natural lesions. IOA above 80% was verified for any scenario. SUV metrics showed a good agreement between synthetic and natural lesions. Conclusions ISL proved relevant to evaluate performance differences between PET scanners. Using these synthetically modified clinical images, we can produce a controlled ground truth in a realistic anatomical model and exploit the potential of PET scanner for clinical purposes.