A novel hyperparameter search approach for accuracy and simplicity in disease prediction risk scoring

Abstract

Abstract Objective Develop a novel technique to identify an optimal number of regression units corresponding to a single risk point, while creating risk scoring systems from logistic regression-based disease predictive models. The optimal value of this hyperparameter balances simplicity and accuracy, yielding risk scores of small scale and high accuracy for patient risk stratification. Materials and Methods The proposed technique applies an adapted line search across all potential hyperparameter values. Additionally, DeLong test is integrated to ensure the selected value produces an accuracy insignificantly different from the best achievable risk score accuracy. We assessed the approach through two case studies predicting diabetic retinopathy (DR) within six months and hip fracture readmissions (HFR) within 30 days, involving cohorts of 90 400 diabetic patients and 18 065 hip fracture patients. Results Our scores achieve accuracies insignificantly different from those obtained by existing approaches, reaching AUROCs of 0.803 and 0.645 for DR and HFR predictions, respectively. Regarding the scale, our scores ranged 0-53 for DR and 0-15 for HFR, while scores produced by existing methods frequently spanned hundreds or thousands. Discussion According to the assessment, our risk scores offer simple and accurate predictions for diseases. Furthermore, our new DR score provides a competitive alternative to state-of-the-art risk scores for DR, while our HFR case study presents the first risk score for this condition. Conclusion Our technique offers a generalizable framework for crafting precise risk scores of compact scales, addressing the demand for user-friendly and effective risk stratification tool in healthcare.