Evaluation of Mental Health Mobile Applications

Abstract

Background. Mental health mobile applications (apps) have the potential to expand the provision of mental health and wellness services to traditionally underserved populations. There is a lack of guidance on how to choose wisely from the thousands of mental health apps without clear evidence of safety, efficacy, and consumer protections. Purpose. This Technical Brief proposes a framework to assess mental health mobile applications with the aim to facilitate selection of apps. The results of applying the framework will yield summary statements on the strengths and limitations of the apps and are intended for use by providers and patients/caregivers. Methods. We reviewed systematic reviews of mental health apps and reviewed published and gray literature on mental health app frameworks, and we conducted four Key Informant group discussions to identify gaps in existing mental health frameworks and key framework criteria. These reviews and discussions informed the development of a draft framework to assess mental health apps. Iterative testing and refinement of the framework was done in seven successive rounds through double application of the framework to a total of 45 apps. Items in the framework with an interrater reliability under 90 percent were discussed among the evaluation team for revisions of the framework or guidance. Findings. Our review of the existing frameworks identified gaps in the assessment of risks that users may face from apps, such as privacy and security disclosures and regulatory safeguards to protect the users. Key Informant discussions identified priority criteria to include in the framework, including safety and efficacy of mental health apps. We developed the Framework to Assist Stakeholders in Technology Evaluation for Recovery (FASTER) to Mental Health and Wellness and it comprises three sections: Section 1. Risks and Mitigation Strategies, assesses the integrity and risk profile of the app; Section 2. Function, focuses on descriptive aspects related to accessibility, costs, organizational credibility, evidence and clinical foundation, privacy/security, usability, functions for remote monitoring of the user, access to crisis services, and artificial intelligence (AI); and Section 3. Mental Health App Features, focuses on specific mental health app features, such as journaling and mood tracking. Conclusion. FASTER may be used to help appraise and select mental health mobile apps. Future application, testing, and refinements may be required to determine the framework’s suitability and reliability across multiple mental health conditions, as well as to account for the rapidly expanding applications of AI, gamification, and other new technology approaches.