Unit Response and Costs in Web Versus Face-To-Face Data Collection: Comparison of Two Cross-sectional Health Surveys

Abstract

Background Potential is seen in web data collection for population health surveys due to its combined cost-effectiveness, implementation ease, and increased internet penetration. Nonetheless, web modes may lead to lower and more selective unit response than traditional modes, and this may increase bias in the measured indicators. Objective This research assesses the unit response and costs of a web study versus face-to-face (F2F) study. Methods Alongside the Belgian Health Interview Survey by F2F edition 2018 (BHISF2F; net sample used: 3316), a web survey (Belgian Health Interview Survey by Web [BHISWEB]; net sample used: 1010) was organized. Sociodemographic data on invited individuals was obtained from the national register and census linkages. Unit response rates considering the different sampling probabilities of both surveys were calculated. Logistic regression analyses examined the association between mode system and sociodemographic characteristics for unit nonresponse. The costs per completed web questionnaire were compared with the costs for a completed F2F questionnaire. Results The unit response rate is lower in BHISWEB (18.0%) versus BHISF2F (43.1%). A lower response rate was observed for the web survey among all sociodemographic groups, but the difference was higher among people aged 65 years and older (15.4% vs 45.1%), lower educated people (10.9% vs 38.0%), people with a non-Belgian European nationality (11.4% vs 40.7%), people with a non-European nationality (7.2% vs 38.0%), people living alone (12.6% vs 40.5%), and people living in the Brussels-Capital (12.2% vs 41.8%) region. The sociodemographic characteristics associated with nonresponse are not the same in the 2 studies. Having another European (OR 1.60, 95% CI 1.20-2.13) or non-European nationality (OR 2.57, 95% CI 1.79-3.70) compared to a Belgian nationality and living in the Brussels-Capital (OR 1.72, 95% CI 1.41-2.10) or Walloon (OR 1.47, 95% CI 1.15-1.87) regions compared to the Flemish region are associated with a higher nonresponse only in the BHISWEB study. In BHISF2F, younger people (OR 1.31, 95% CI 1.11-1.54) are more likely to be nonrespondents than older people, and this was not the case in BHISWEB. In both studies, lower educated people have a higher probability of being nonrespondent, but this effect is more pronounced in BHISWEB (low vs high education level: Web, OR 2.71, 95% CI 2.21-3.39 and F2F OR 1.70, 95% CI 1.48-1.95). The BHISWEB study had a considerable advantage; the cost per completed questionnaire was almost 3 times lower (€41 [US $48]) compared with F2F data collection (€111 [US $131]). Conclusions The F2F unit response rate was generally higher, yet for certain groups the difference between web and F2F was more limited. Web data collection has a considerable cost advantage. It is therefore worth experimenting with adaptive mixed-mode designs to optimize financial resources without increasing selection bias (eg, only inviting sociodemographic groups who are keener to participate online for web surveys while continuing to focus on increasing F2F response rates for other groups).