Mining the Influencing Factors and Their Asymmetrical Effects of mHealth Sleep App User Satisfaction From Real-world User-Generated Reviews: Content Analysis and Topic Modeling

Abstract

Background Sleep disorders are a global challenge, affecting a quarter of the global population. Mobile health (mHealth) sleep apps are a potential solution, but 25% of users stop using them after a single use. User satisfaction had a significant impact on continued use intention. Objective This China-US comparison study aimed to mine the topics discussed in user-generated reviews of mHealth sleep apps, assess the effects of the topics on user satisfaction and dissatisfaction with these apps, and provide suggestions for improving users’ intentions to continue using mHealth sleep apps. Methods An unsupervised clustering technique was used to identify the topics discussed in user reviews of mHealth sleep apps. On the basis of the two-factor theory, the Tobit model was used to explore the effect of each topic on user satisfaction and dissatisfaction, and differences in the effects were analyzed using the Wald test. Results A total of 488,071 user reviews of 10 mainstream sleep apps were collected, including 267,589 (54.8%) American user reviews and 220,482 (45.2%) Chinese user reviews. The user satisfaction rates of sleep apps were poor (China: 56.58% vs the United States: 45.87%). We identified 14 topics in the user-generated reviews for each country. In the Chinese data, 13 topics had a significant effect on the positive deviation (PD) and negative deviation (ND) of user satisfaction. The 2 variables (PD and ND) were defined by the difference between the user rating and the overall rating of the app in the app store. Among these topics, the app’s sound recording function (β=1.026; P=.004) had the largest positive effect on the PD of user satisfaction, and the topic with the largest positive effect on the ND of user satisfaction was the sleep improvement effect of the app (β=1.185; P<.001). In the American data, all 14 topics had a significant effect on the PD and ND of user satisfaction. Among these, the topic with the largest positive effect on the ND of user satisfaction was the app’s sleep promotion effect (β=1.389; P<.001), whereas the app’s sleep improvement effect (β=1.168; P<.001) had the largest positive effect on the PD of user satisfaction. The Wald test showed that there were significant differences in the PD and ND models of user satisfaction in both countries (all P<.05), indicating that the influencing factors of user satisfaction with mHealth sleep apps were asymmetrical. Using the China-US comparison, hygiene factors (ie, stability, compatibility, cost, and sleep monitoring function) and 2 motivation factors (ie, sleep suggestion function and sleep promotion effects) of sleep apps were identified. Conclusions By distinguishing between the hygiene and motivation factors, the use of sleep apps in the real world can be effectively promoted.