Hybrid Diagnosis Models for Autism Patients Based on Medical and Sociodemographic Features Using Machine Learning and Multicriteria Decision-Making (MCDM) Techniques: An Evaluation and Benchmarking Framework

Abstract

Background and Contexts. Autism spectrum disorder (ASD) is difficult to diagnose, prompting researchers to increase their efforts to find the best diagnosis by introducing machine learning (ML). Recently, several available challenges and issues have been highlighted for the diagnosis of ASD. High consideration must be taken into the feature selection (FS) approaches and classification process simultaneously by using medical tests and sociodemographic characteristic features in autism diagnostic. The constructed ML models neglected the importance of medical tests and sociodemographic features in a training and evaluation dataset, especially since some features have different contributions to the processing data and possess more relevancies to the classification information than others. However, the role of the physician’s experience towards feature contributions remains limited. In addition, the presence of many evaluation criteria, criteria trade-offs, and criteria importance categorize the evaluation and benchmarking of diagnosis ML models concerning the intersection between FS approaches and ML classification methods given under complex multicriteria decision-making (MCDM) problems. To date, no study has presented an evaluation framework for benchmarking the best hybrid diagnosis models to classify autism patients’ emergency levels considering multicriteria evaluation solutions. Method. The three-phase framework integrated the MCDM and ML to develop the diagnosis models and evaluate and benchmark the best. Firstly, the new ASD-dataset-combined medical tests and sociodemographic characteristic features is identified and preprocessed. Secondly, developing the hybrid diagnosis models using the intersection process between three FS techniques and five ML algorithms introduces 15 models. The selected medical tests and sociodemographic features from each FS technique are weighted before feeding the five ML algorithms using the fuzzy-weighted zero-inconsistency (FWZIC) method based on four psychiatry experts. Thirdly, (i) formulate a dynamic decision matrix for all developed models based on seven evaluation metrics, including classification accuracy, precision, F1 score, recall, test time, train time, and AUC. (ii) The fuzzy decision by opinion score method (FDOSM) is used to evaluate and benchmark the 15 models concerning the seven evaluation metrics. Results. Results reveal that (i) the three FS techniques have obtained a size different from the others in the number of the selected features; the sets were 39, 38, and 41 out of 48 features. Each set has its weights constructed by FWIZC. Considered sociodemographic features have been mostly selected more than medical tests within FS techniques. (ii) The first three best hybrid models were “ReF-decision tree,” “IG-decision tree,” and “Chi2-decision tree,” with score values 0.15714, 0.17539, and 0.29444. The best diagnosis model (ReF-decision tree) has obtained 0.4190, 0.0030, 0.9946, 0.9902, 0.9902, 0.9902, 0.9902, and 0.9951 for the C1=train time, C2=test time, C3=AUC, C4=CA, C5=F1 score, C6=precision, and C7=recall, respectively. The developed framework would be beneficial in advancing, accelerating, and selecting diagnosis tools in therapy with ASD. The selected model can identify severity as light, medium, or intense based on medical tests and sociodemographic weighted features.