Evaluation of a Mobile Device Survey System for Behavioral Risk Factors (SHAPE): App Development and Usability Study

Abstract

Background Risk factors, including limited exercise, poor sleep, smoking, and alcohol and drug use, if mitigated early, can improve long-term health. Risk prevalence has traditionally been measured using methods that now have diminished participation rates. With >75% of American citizens owning smartphones, new data collection methods using mobile apps can be evaluated. Objective The objective of our study was to describe the development, implementation, and evaluation of a mobile device–based survey system for behavioral risk assessment. Specifically, we evaluated its feasibility, usability, acceptability, and validity. Methods We enrolled 536 students from 3 Vermont State Colleges. Iterative mobile app development incorporated focus groups, extensive testing, and the following 4 app versions: iOS standard, iOS gamified, Android standard, and Android gamified. We aimed to capture survey data, paradata, and ambient data such as geolocation. Using 3 separate surveys, we asked a total of 27 questions that included demographic characteristics, behavioral health, and questions regarding the app’s usability and survey process. Results Planned enrollment was exceeded in just a few days. There were 1392 “hits” to the landing page where the app could be downloaded. Excluding known project testers and others not part of the study population, 670 participants downloadeded the SHAPE app. Of those, 94.9% of participants (636/670) agreed to participate by providing in-app consent. Of the 636 who provided consent, 84.3% (536/636) were deemed eligible for the study. The majority of eligible respondents completed the initial survey (459/536, 85.6%), whereas 29.9% (160/536) completed the second survey and 28.5% (153/536) completed the third survey. The SHAPE survey obtained 414 participants on the behavioral risk items in survey 1, which is nearly double the 209 participants who completed the traditional Vermont College Health Survey in 2014. SHAPE survey responses were consistent with the traditionally collected Vermont College Health Survey data. Conclusions This study provides data highlighting the potential for mobile apps to improve population-based health, including an assessment of recruitment methods, burden and response rapidity, and future adaptations. Although gamification and monetary rewards were relatively unimportant to this study population, item response theory may be technologically feasible to reduce individual survey burden. Additional data collected by smartphones, such as geolocation, could be important in additional analysis, such as neighborhood characteristics and their impact on behavioral risk factors. Mobile tools that offer rapid adaptation for specific populations may improve research data collection for primary prevention and could be used to improve engagement and health outcomes.