Adapting and Evaluating an Artificial Intelligence-Based Chatbot through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial Study (the MARVIN Chatbots) (Preprint)

Abstract

Self-management interventions are emphasized as a way to optimize health outcomes and care, but they necessitate supporting patients to access timely and reliable health information, which remains a challenge. In the realm of digital health, artificial intelligence-based chatbots capable of conversing with users in natural language have emerged as promising tools. However, it is imperative to conduct further research on their implementation. Besides, inclusive digital health research and responsive AI integration into healthcare necessitates active and sustained patients and stakeholder engagement.

This manuscript presents the master protocol of the MARVIN chatbots study which has four objectives: 1) co-construct tailored AI chatbots for different healthcare settings; 2) assess their usability in a small participant sample context; 3) measure their implementation outcomes (usability, acceptability, appropriateness, adoption, and fidelity) in a large sample context; and 4) evaluate the impact of patient and stakeholder partnerships on their development. This adaptive platform trial consists of multiple parallel individual chatbot sub-studies sharing common objectives, with Objective 1 through 3 to be completed sequentially, and Objective 4 to be assessed throughout.

The study will recruit patients and healthcare professionals from the McGill University Health Centre and the Centre hospitalier de l’Université de Montréal (both in Montreal, Canada). Four needs assessment focus groups with 20 participants and six co-construction workshops with a co-construction design committee will first be conducted to develop and test chatbots adapted to the relevant healthcare context. Thirty participants will then interact with MARVIN for three weeks and assess its usability through a survey and three focus groups. Positive usability outcomes will lead to public access of the chatbot for a one-year real world implementation study. Questionnaires will be administered online to 150 participants to measure usability, acceptability, and appropriateness, while meta-use data will inform adoption and fidelity. Following completion of each objective, focus groups will be conducted with the co-construction design committee to better understand stakeholders’ perspectives on their engagement in research.

From July 2022 to October 2023, this master protocol led to four sub-studies: 1) MARVIN for HIV (large-scale implementation expected to begin in mid-2024); 2) MARVIN “Pharma” to support community pharmacist in providing HIV care (usability study planned for mid-2024); 3) MARVINA for breast cancer, and 4) MARVIN Champ for pediatric infectious conditions (both in preparation with development to begin in early-2024).

The development and adaptation of the MARVIN chatbots is expected to improve patient self-management and healthcare efficiency. This master protocol comprehensively examines the implementation outcomes of chatbot interventions for patients and healthcare professionals. It will also contribute to best practice recommendations for patient and stakeholder engagement in digital health research. With appropriate design, this protocol can sustain long-term innovation translation, and deliver timely interventions to advance patient-centered personalized medicine.

ClinicalTrials.gov identifier: NCT05789901 https://classic.clinicaltrials.gov/ct2/show/NCT05789901