Recommendations From a Descriptive Evaluation to Improve Screening Procedures for Web-Based Studies With Couples: Cross-Sectional Study (Preprint)

Abstract

Although there are a number of advantages to using the internet to recruit and enroll participants into Web-based research studies, these advantages hinge on data validity. In response to this concern, researchers have provided recommendations for how best to screen for fraudulent survey entries and to handle potentially invalid responses. Yet, the majority of this previous work focuses on screening (ie, verification that individual met the inclusion criteria) and validating data from 1 individual, and not from 2 people who are in a dyadic relationship with one another (eg, same-sex male couple; mother and daughter). Although many of the same data validation and screening recommendations for Web-based studies with individual participants can be used with dyads, there are differences and challenges that need to be considered.

This paper aimed to describe the methods used to verify and validate couples’ relationships and data from a Web-based research study, as well as the associated lessons learned for application toward future Web-based studies involving the screening and enrollment of couples with dyadic data collection.

We conducted a descriptive evaluation of the procedures and associated benchmarks (ie, decision rules) used to verify couples’ relationships and validate whether data uniquely came from each partner of the couple. Data came from a large convenience sample of same-sex male couples in the United States, who were recruited through social media venues for a Web-based, mixed methods HIV prevention research study.

Among the 3815 individuals who initiated eligibility screening, 1536 paired individuals (ie, data from both partners of a dyad) were assessed for relationship verification; all passed this benchmark. For data validation, 450 paired individuals (225 dyads) were identified as fraudulent and failed this benchmark, resulting in a total sample size of 1086 paired participants representing 543 same-sex male couples who were enrolled. The lessons learned from the procedures used to screen couples for this Web-based research study have led us to identify and describe four areas that warrant careful attention: (1) creation of new and replacement of certain relationship verification items, (2) identification of resources needed relative to using a manual or electronic approach for screening, (3) examination of approaches to link and identify both partners of the couple, and (4) handling of <i>bots</i>.

The screening items and associated rules used to verify and validate couples’ relationships and data worked yet required extensive resources to implement. New or updating some items to verify a couple’s relationship may be beneficial for future studies. The procedures used to link and identify whether both partners were coupled also worked, yet they call into question whether new approaches are possible to help increase linkage, suggesting the need for further inquiry.