Emotion Forecasting: A Transformer-Based Approach (Preprint)

Abstract

Monitoring the emotional states of psychiatric patients has always been challenging due to the non-continuous nature of clinical assessments, the effect of being in a healthcare environment, and the inherent subjectivity of existing evaluation instruments. However, mental states in psychiatric disorders exhibit significant variability over time, making real-time monitoring crucial for preventing risk situations and ensuring appropriate treatment.

Our objective is to leverage new technologies and deep learning techniques to enable a more objective, real-time monitoring of patients. This will be achieved by passively monitoring variables like step count, patient location, and sleep patterns using mobile devices. We aim to predict patient self-reports and detect sudden variations in their emotional valence, identifying situations that may require clinical intervention.

Data for this project are registered with the Evidence-Based Behavior (eB2) MindCare mobile application, where both passively and self-reported variables are recorded from patients. We utilize daily summaries of these variables. We implement imputation methods based on hidden Markov model (HMM) to address missing data and transformer deep neural networks for time-series forecasting. Finally, classification algorithms are applied to predict several variables, including emotional state and responses to the Patient Health Questionnaire (PHQ-9).

Through real-time patient monitoring, we demonstrated the ability to accurately predict their emotional state, obtaining an accuracy of 0.93 and 0.98 of receiver operating characteristic (ROC) area under the curve (AUC) for emotional valence classification with an XGBoost classifier and anticipate emotional state changes (ROC AUC of 0.87 for change detection one day in advance). Additionally, we showed the feasibility of forecasting general responses to the PHQ-9 questionnaire. Especially good results were obtained for the score prediction of certain questions. For instance, in the case of question 9, related to suicidal ideation, we obtained an accuracy of 0.9 and ROC AUC of 0.768 in predicting the following day’s response. Secondly, from a methodological perspective, we illustrate the enhanced stability of multivariate time-series forecasting when combining HMM pre-processing with a transformer model, as opposed to other time-series forecasting methods, such as the Recurrent Neural Network or the Long Short- Term Memory cells. Concretely, we exploit the capabilities offered by attention mechanisms to capture longer time dependencies.

From a methodological perspective, we found out that the stability of multivariate time-series forecasting improved when combining hidden Markov model pre-processing with a transformer model, as opposed to other time-series forecasting methods (RNN, LSTM...), leveraging the attention mechanisms to capture longer time dependencies and gain interpretability. We show the potential to assess the emotional state of a patient and the scores of psychiatric questionnaires from passive variables in advance. This offers a real real-time monitoring of patients and hence better risk detection and treatment adjustment.