Learning Dynamic Treatment Strategies for Coronary Heart Diseases by Artificial Intelligence: Real-world Data Driven Study (Preprint)

Abstract

Coronary heart disease (CHD) had become the world’s leading cause of death, and one of the worldwide most serious epidemic diseases. CHD was characterized by urgency, danger and severity, to make dynamic treatment strategies for CHD patients in the ICUs are of great importance. However, few works had done on this.

We aimed at building and validating an artificial intelligence (AI) model to learn dynamic treatment strategies for CHD patients with the goal of improving patient outcome and learning the best practice from clinicians at the same time.

We formed the treatment strategy as a sequential decision problem, and applied an AI framework of combining supervised learning and reinforcement learning (SRL) to learn a recommendation model from the real-world EHRs that took the patient’s diagnosis and evolving health status as input, and gave a treatment recommendation in the form of whether or not to take the specific drugs. The experiments were conducted by leveraging a real-world ICU database with 13,762 hospital admissions diagnosed with CHD, and the long-short-term memory (LSTM) was adopted to track the long term observed states of CHD patients. We compared the AI model with its supervised learning (SL) prototype and reinforcement learning (RL) prototype on the performance of reducing estimated in-hospital mortality and the Jaccard similarity with clinician decision by 5-fold cross validation and independent test. We used the random forest algorithm to calculate the feature importance of both the clinician policy and the AI policy, so as to illustrate the interpretability of the AI model. A case study was conducted to see the similarities and differences between the AI recommend treatment actions and the clinician’s actual treatment decisions.

The AI model with 40% reinforcement learning and 60% supervised learning could help reducing estimated in-hospital mortality by 3.13% and 0.81% on trajectory and state-wise respectively, and ensure the Jaccard similarity (0.3110) close to that of the supervised learning (0.3432). We observed that the smallest treatment action difference between AI policy and clinician decision was associated with the best survival rates, the further the prescribed drugs away from the AI recommended treatment, the worse the outcome. The case study showed that the similarity between clinician decision and AI policy on the survival case was high, while on the expired case was much smaller.

We applied an AI model of SRL-LSTM to learn dynamic treatment strategies for CHD patients in ICUs by using real-world EHRs, and it was proved to have the ability to help improving the outcome of CHD patients through its RL part, as well as learning the best practice of the clinicians by the SL part.