Assessing the Usage and Usability of a Mental Health Advice Telephone Service in Uganda: A Mixed Methods Study (Preprint)

Abstract

Harnessing mobile health (mHealth) solutions could improve the delivery of mental health services and reduce their impact in Uganda and similar low-resource settings. However, successful adoption requires that mHealth solutions have good usability. We have previously implemented a telephone service to provide mental health information and advice in English and Luganda using an automated (interactive voice response) system and linkage to live agents that include mental healthcare workers and peer support workers.

To assess the usage and usability of this mental health telephone service.

We obtained usage data from the system call logs for 18 months to study the call volumes and trends. We then surveyed callers to obtain their characteristics and assess usability using the telehealth usability questionnaire. Additionally, call recordings were assessed for conversation quality using the telephone nursing dialogue process by three independent healthcare professionals, and correlations between quality and usability aspects were investigated.

In 18 months, 2863 meaningful calls (i.e., went past the welcome message) from 1125 unique telephone numbers were made to the system. 1153 (40.3%) stopped at the pre-recorded interactive voice response information, and 1710 (59.7%) chose to speak to an agent; of which 1292 (75.6%) were answered, 393 (23.0%) went to voicemail and were returned in the following working days, and 25 (1.5%) were not answered. Usage was generally sustained over time, with spikes in calls corresponding to marketing events. The survey (n=240) revealed that most callers were caregivers of mental health patients (60%) or the general public (19.2%), and few were mental health patients (18.3%). Additionally, majority were male (59.6%), spoke English (75.0%), had post-secondary education (68.3%), lived one hour or less from Butabika Hospital (77.9%), and were aged 25-44 (66.7%). The overall usability score for the system was 4.12 on a 5-point scale, significantly higher than the recommended target usability score of 4 (p-value = 0.006). Mean scores for usability components ranged from 3.66 for Reliability, and 4.41 for Ease of use, and all components but Reliability were higher than 4 or within its confidence interval. Usability scores were higher for Luganda than English speakers, but there was no association with other participant characteristics such as sex, distance from the hospital, age, marital status, duration of symptoms, or treatment status. The quality of call conversations (n=50) was 4.35 out of 5 and significantly correlated with usability (Pearson r = 0.34, p<0.05).

We found sustained usage of the mental health telephone service, with good user experience and high satisfaction across different user characteristics. Mobile health solutions like these should be embraced and replicated to improve the delivery of health services in Uganda and similar low-resource settings.