Predicting One-Year Outcome in First Episode Psychosis using Machine Learning

Abstract

Lay SummaryEvidence before this studyOur knowledge of factors which predict outcome in first episode psychosis (FEP) is incomplete. Poor premorbid adjustment, history of developmental disorder, symptom severity at baseline and duration of untreated psychosis are the most replicated predictors of poor clinical, functional, cognitive, and biological outcomes. Yet, such group level differences are not always replicated in individuals, nor can observational results be clearly equated with causation. Advanced machine learning techniques have potential to revolutionise medicine by looking at causation and the prediction of individual patient outcome. Within psychiatry, Koutsouleris et al employed machine learning to predict 4- and 52-week functional outcome in FEP to a 75% and 73.8% test-fold balanced accuracy on repeated nested internal cross-validation. The authors suggest that before employing a machine learning model “into real-world care, replication is needed in external first episode samples”.Added value of this studyWe believe our study to be the first externally validated evidence, in a temporally and geographically independent cohort, for predictive modelling in FEP at an individual patient level. Our results demonstrate the ability to predict both symptom remission and functioning (in employment, education or training (EET)) at one-year. The performance of our EET model was particularly robust, with an ability to accurately predict the one-year EET outcome in more than 85% of patients. Regularised regression results in sparse models which are uniquely interpretable and identify meaningful predictors of recovery including specific individual PANSS items, and social support. This builds on existing studies of group-level differences and the elegant work of Koutsouleris et al.Implications of all the available evidenceWe have demonstrated the externally validated ability to accurately predict one-year symptomatic and functional status in individual patients with FEP. External validation in a plausibly related temporally and geographically distinct population assesses model transportability to an untested situation rather than simply reproducibility alone. We propose that our results represent important and exciting progress in unlocking the potential of predictive modelling in psychiatric illness. The next step prior to implementation into routine clinical practice would be to establish whether, by the accurate identification of individuals who will have poor outcomes, we can meaningful intervene to improve their prognosis.AbstractBackgroundEarly illness course correlates with long-term outcome in psychosis. Accurate prediction could allow more focused intervention. Earlier intervention corresponds to significantly better symptomatic and functional outcomes.We use routinely collected baseline demographic and clinical characteristics to predict employment, education or training (EET) status, and symptom remission in patients with first episode psychosis (FEP) at 1 year.Methods83 FEP patients were recruited from National Health Service (NHS) Glasgow between 2011 and 2014 to a 24-month prospective cohort study with regular assessment of demographic and psychometric measures. An external independent cohort of 79 FEP patients were recruited from NHS Glasgow and Edinburgh during a 12-month study between 2006 and 2009.Elastic net regularised logistic regression models were built to predict binary EET status, period and point remission outcomes at 1 year on 83 Glasgow patients (training dataset). Models were externally validated on an independent dataset of 79 patients from Glasgow and Edinburgh (validation dataset). Only baseline predictors shared across both cohorts were made available for model training and validation.OutcomesAfter excluding participants with missing outcomes, models were built on the training dataset for EET status, period and point remission outcomes and externally validated on the validation dataset. Models predicted EET status, period and point remission with ROC area under curve (AUC) performances of 0.876 (95%CI: 0.864, 0.887), 0.630 (95%CI: 0.612, 0.647) and 0.652 (95%CI: 0.635, 0.670) respectively. Positive predictors of EET included baseline EET and living with spouse/children. Negative predictors included higher PANSS suspiciousness, hostility and delusions scores. Positive predictors for symptom remission included living with spouse/children, and affective symptoms on the Positive and Negative Syndrome Scale (PANSS). Negative predictors of remission included passive social withdrawal symptoms on PANSS.InterpretationUsing advanced statistical machine learning techniques, we provide the first externally validated evidence for the ability to predict 1-year EET status and symptom remission in FEP patients.FundingThe authors acknowledge financial support from NHS Research Scotland, the Chief Scientist Office, the Wellcome Trust, and the Scottish Mental Health Research Network.