Best practices for engagement in remote participatory design: Mixed method analysis of four remote studies with family caregivers (Preprint)

Abstract

Digital health interventions are a promising method for delivering timely support to under-resourced family caregivers. The uptake of digital health interventions among caregivers may be improved by engaging caregivers in their design through Participatory design (PD). Recent years have seen a shift towards conducting PD remotely, which may enable participation by previously hard-to-reach groups. However, little is known regarding how best to facilitate engagement in remote PD among family caregivers.

The objective of the present study was to 1) understand the context, quality, and outcomes of family caregivers’ engagement experiences in remote PD, and 2) learn which aspects of the observed PD approach facilitated engagement or need to be improved going forward.

We analyzed qualitative and quantitative data from evaluation and reflection surveys completed by research partners and community partners (family caregivers) across four separate remote PD studies conducted between 2021-2023. Each study focused on building digital health interventions for family caregivers. For each study, community partners met with research partners for 4-5 design sessions over the course of 6 months. After each design session, community partners completed an evaluation survey. Additionally, after one of four studies, research and community partners completed a reflective survey and interview. Descriptive statistics were used to summarize quantitative evaluation and reflection survey data, while thematic analysis was used to understand all qualitative evaluation and reflection survey data.

The average effectiveness and satisfaction ratings for each session ranged between 4 and 5 on a five-point scale. Qualitative data relating to the Engagement Context identified that the identities of partners, the technological context of remote PD, and partners' understanding of the project and their role all influenced engagement. Within the domain of Engagement Quality, relationship building and co-learning; satisfaction with pre-work, design activities, time allotted, and the final prototype; and inclusivity and the distribution of influence contributed to partners' experience of engagement. Data pertaining to Partner Outcomes indicated that partners felt ongoing interest in the project after its conclusion, felt gratitude for participation, and gained a sense of meaning and self- esteem from engaging in remote PD.

These results point to high satisfaction with remote PD processes and few losses specific to remote PD. Results also demonstrate specific ways in which processes can be changed to improve partner engagement and outcomes. Community partners should be involved from study inception in defining the problem to be solved, the approach to be used, and their roles within the project. Throughout the design process, virtual tools may be used to check partners' satisfaction with design processes and perceptions of inclusivity and power sharing. Emphasis should be placed on increasing psychosocial benefits of engagement (e.g., sense of community, purpose) and increasing opportunities to participate in disseminating findings and future studies.