Viability of Web-based Respondent Driven Sampling with the goal of population inference in high-income countries; lessons learned from a process evaluation of WEB-RDS among Men who have Sex with Men in Belgium (Preprint)

Abstract

Obtaining a representative sample is a substantial challenge when undertaking health research among hidden and hard-to-reach populations such as men who have sex with men (MSM). Web based Respondent Driven Sampling (WEB RDS) was developed to overcome such sampling challenges and to create population estimates based on network and sampling characteristics. Despite a decade of research in low- and middle-income countries it remains unclear whether WEB RDS is suitable for sampling hidden populations in high-income countries.

This study’s objective was to evaluate how viable WEB RDS methodology is for obtaining a nationwide sample of MSM, suitable for population inference of sexual health characteristics, in a high income setting such as Belgium.

We adapted the MRC (Medical Resource Council) process evaluation framework for interventions, to evaluate an empirical WEB RDS. Viewing ‘WEB RDS’ as a complex intervention with respondent driven recruitment as the aim, we evaluated indicators of context, implementation, mechanism of impact and performance. We analysed the data using a mixed-methods approach that integrated findings from quantitative analysis such as RDS diagnostics and qualitative thematic analysis..

Sampling did not reach a sufficient sample size (n=193) to compensate for an RDS design effect of 3 and the number of recruitment waves was low (waves=7). Homophily values (age=1.51, education=0.88, place of residence =1.13) signal a high degree of clustering in the network, and qualitative data indicates MSM in Belgium are likely not one network at the national level. Bottleneck plots indicate that many more waves of recruitment would be needed for population estimates to become independent of the initial respondents, yet we found difficulties inherent to producing more waves in the research context. MSM and MSM community organization representatives indicated that, in Belgium, MSM are over-researched, with low motivation for the topic of sexual health and that digital etiquette dictates not sharing survey links. A moderate reward of 10-30 euro’s with a dual incentive structure was insufficient to overcome these barriers.

In conclusion, our study shows that WEB RDS with a moderate incentive is not a viable sampling strategy for obtaining national population estimates of sexual health traits of MSM in Belgium. The study emphasizes the need for understanding MSM research motivation and topic saliency, exploring optimal incentive levels in a high income context, addressing over-research, and considering alternative methods, such as offline RDS or hybrid approaches, to capture diverse subpopulations. Additionally, the study highlights the importance of online etiquette and geographical clustering considerations for successful WEB RDS implementation. Finally, the study showcases the utility of the adapted MRC framework for evaluating WEB RDS methodology.