Design, Implementation, and Evaluation of the Computer-aided Clinical Decision Support System based on Learning-to-Rank: Collaboration between physicians and machine learning in the differential diagnosis process

Abstract

Abstract OBJECTIVES We are developing the Clinical Decision Support System (CDSS) based on Learning-to-Rank (LTR). The main objectives are 1) Supporting differential diagnoses by internists and general practitioners and 2) Preventing diagnostic errors by physicians. The main features are that "A physician inputs a patient's symptoms, findings, and test results to the system, and the system outputs a ranking list of possible diseases." METHOD The software libraries for machine learning and artificial intelligence are TensorFlow and TensorFlow Ranking. The prediction algorithm is LTR with a listwise approach. The ranking metric is NDCG. The loss functions are Approximate NDCG (A-NDCG) and Gumbel Approximate NDCG (G-A-NDCG). We evaluated Machine Learning (ML) performance and Differential Diagnosis (DDx) performance with actual cases. RESULTS ML performance of our system was much higher than that of the conventional system. ML performance using G-A-NDCG was slightly higher than that of A-NDCG. DDx performance of our system was much higher than that of the conventional system. We have shown that CDSS prevents physicians' diagnostic errors due to confirmation bias. CONCLUSIONS We have demonstrated that the CDSS is useful for supporting differential diagnoses and preventing diagnostic errors. We believe that DDx by physicians and LTR have a high affinity. We found that Information Retrieval (IR) and Clinical Decision Support System (CDSS) have much in common (target data, LTR, etc.). We believe that CDSS has the potential to support 1) recall of rare diseases, 2) differential diagnoses for difficult-to-diagnose diseases, and 3) prevention of diagnostic errors. We also believe that our system has the potential for evolution to an Explainable Clinical Decision Support System (X-CDSS).