Abstract
Background
Due to a resurgent debate on the misuse of P values, the Journal of Medical Internet Research is hosting a standing theme issue inviting the reanalysis of (primarily digital health) trial data using a Bayesian framework. This first paper in this series focuses on an electronic screening and brief intervention (eSBI), targeting harmful and hazardous alcohol consumption, which student health care centers across Sweden have routinely administerd to all students during the past decade. The second Alcohol Email Assessment and Feedback Study Dismantling Effectiveness for University Students (AMADEUS-2) trial aimed to assess the effect of the eSBI on alcohol consumption among students who were harmful and hazardous drinkers. A two-arm randomized controlled trial design was employed, randomizing eligible participants to either a waiting list or direct access to an eSBI. Follow-up assessments were conducted 2 months after randomization. Subsequent analysis of the trial followed the conventional null hypothesis approach, and no statistical significance was found between groups at follow-up with respect to the number of standard drinks consumed weekly. However, in an unspecified sensitivity analysis, it was discovered that removing three potential outliers made the difference between the groups significant.
Objective
The objective of this study is to reperform the primary and sensitivity analysis of the AMADEUS-2 trial using a Bayesian framework and to compare the results with those of the original analysis.
Methods
The same regression models used in the original analysis were employed in this reanalysis (negative binomial regression). Model parameters were given uniform priors. Markov chain Monte Carlo was used for Bayesian inference, and posterior probabilities were calculated for prespecified thresholds of interest.
Results
Null hypothesis tests did not identify a statistically significant difference between the intervention and control groups, potentially due to a few extreme data points. The Bayesian analysis indicated a 93.6% probability that there was a difference in grams of alcohol consumed at follow-up between the intervention and control groups and a 71.5% probability that the incidence rate ratio was <0.96. Posterior probabilities increased when excluding three potential outliers, yet such post hoc analyses were not necessary to show the preference toward offering an eSBI to harmful and hazardous drinkers among university students.
Conclusions
The null hypothesis framework relies on point estimates of parameters. P values can therefore swing heavily, depending on a single or few data points alone, casting doubt on the value of the analysis. Bayesian analysis results in a distribution over parameter values and is therefore less sensitive to outliers and extreme values. Results from analyses of trials of interventions where small-to-modest effect sizes are expected can be more robust in a Bayesian framework, making this a potentially better approach for analyzing digital health research data.
Trial Registration
International Standard Randomized Controlled Trial Number (ISRCTN) 02335307; http://www.isrctn.com/ISRCTN02335307