Machine Learning and Feature Selection Applied to SEER Data to Reliably Assess Thyroid Cancer Prognosis

Abstract

AbstractUtilizing historical clinical datasets to guide future treatment choices is beneficial for patients and physicians. Machine learning and feature selection algorithms (namely, Fisher’s discriminant ratio, Kruskal-Wallis’ analysis, and Relief-F) have been combined in this research to analyse a SEER database containing clinical features from de-identified thyroid cancer patients. The data covered 34 unique clinical variables such as patients’ age at diagnosis or information regarding lymph nodes, which were employed to build various novel classifiers to distinguish patients that lived for over 10 years since diagnosis, from those who did not survive at least five years. By properly optimizing supervised neural networks, specifically multilayer perceptrons, using data from large groups of thyroid cancer patients (between 6,756 and 20,344 for different models), we demonstrate that unspecialized and existing medical recording can be reliably turned into power of prediction to help doctors make informed and optimized treatment decisions, as distinguishing patients in terms of prognosis has been achieved with 94.5% accuracy. We also envisage the potential of applying our machine learning strategy to other diseases and purposes such as in designing clinical trials for unmasking the maximum benefits and minimizing risks associated with new drug candidates on given populations.