Evaluation of the Clinical Utility of the Bone Metastases Ensemble Trees for Survival Decision Support Platform (BMETS-DSP): A Case-Based Pilot Assessment

Abstract

PURPOSE The Bone Metastases Ensemble Trees for Survival Decision Support Platform (BMETS-DSP) provides patient-specific survival predictions and evidence-based recommendations to guide multidisciplinary management for symptomatic bone metastases. We assessed the clinical utility of the BMETS-DSP through a pilot prepost design in a simulated clinical environment. METHODS Ten Radiation Oncology physicians reviewed 55 patient cases at two time points: without and then with the use of BMETS-DSP. Assessment included 12-month survival estimate, confidence in and likelihood of sharing estimates with patients, and recommendations for open surgery, systemic therapy, hospice referral, and radiotherapy (RT) regimen. Paired statistics compared pre- versus post-DSP outcomes. Reported statistical significance is P < .05. RESULTS Pre- versus post-DSP, overestimation of true minus estimated survival time was significantly reduced (mean difference –2.1 [standard deviation 4.1] v –1 month [standard deviation 3.5]). Prediction accuracy was significantly improved at cut points of < 3 (72 v 79%), ≤ 6 (64 v 71%), and ≥ 12 months (70 v 81%). Median ratings of confidence in and likelihood of sharing prognosis significantly increased. Significantly greater concordance was seen in matching use of 1-fraction RT with the true survival < 3 months (70 v 76%) and < 10-fraction RT with the true survival < 12 months (55 v 62%) and appropriate use of open surgery (47% v 53%), without significant changes in selection of hospice referral or systemic therapy. CONCLUSION This pilot study demonstrates that BMETS-DSP significantly improved physician survival estimation accuracy, prognostic confidence, likelihood of sharing prognosis, and use of prognosis-appropriate RT regimens in the care of symptomatic bone metastases, supporting future multi-institutional validation of the platform.