An artificial intelligence-based virtual human avatar application to assess the mental health of healthcare professionals: A validation study (Preprint)

Abstract

Many healthcare workers experience high levels of stress, mental health issues and burnout, yet are less likely than average to seek mental health support. Given this challenge, approaches that promote non-stigmatizing access to assessments of mental health for this population are greatly needed to promote both self-awareness and connection to care. Previous researchers have indicated that when users engaged in dialog with a virtual human agent (VHA), they disclosed more information about their mental health compared to similar interactions with a live human or human-as-avatar assessor. However, applications using this approach have not yet been validated and tested for use in the real world with healthcare professionals.

Given this unmet need, an application called “BeCalm” was developed to conduct psychological assessments using conversational artificial intelligence (AI) and user interaction with a digital, human-like VHA. The BeCalm application allows for both spoken and written (“chat” based) communication. This pilot study aimed to measure the user experience, acceptability, and convergent validity of the BeCalm application.

A cross-sectional one-arm study of the BeCalm application was conducted with 38 healthcare workers (age M = 31.87, SD = 11.28; 84% biologically female).

Qualitative interviews indicated that the apparent interpersonal connection with the VHA was the most appealing aspect of the BeCalm application, with participants describing the VHA as warm and non-judgmental. The second most highly rated aspect of the application was the resources, summary, and psychoeducation provided at the end of the assessment. Many participants reported that they learned something new from the summary and psychoeducation, and the resources were described as concrete, actionable, and “not something I could just google.” There was also strong convergent validity between the BeCalm assessments and conventional mental health measures across several symptom domains (r’s ranged from .101 to .766), including occupational burnout, work-related trauma, work satisfaction, anxiety, mood, and loneliness. Notably, spoken verbal interactions with the VHA elicited longer participant responses by an average of 60 characters compared to chat interactions. Interestingly, participants with higher levels of loneliness tended to provide lengthier answers across all domains, potentially suggesting a desire for deeper engagement with the VHA.

These results demonstrate the ability of BeCalm to provide a valid mental health assessment and resources for an underserved, at-risk population, through interactive technology and personalized feedback. The BeCalm application offers a user-friendly, scalable method for assessing healthcare workers’ mental health that could lead to greater self-awareness and behavioral change. Future studies will determine whether BeCalm can improve mental health outcomes among healthcare professionals and other at-risk populations.