Can a Non-Probabilistic Online Panel Achieve Question Quality Similar to that of the European Social Survey?

Abstract

Recently, Revilla and Saris (2012) showed, using data from the Netherlands, that the quality of responses (product of reliability and validity) in a probability-based online panel (LISS) can be similar to those from face-to-face surveys (European Social Survey round 4). However, most online panels select their members in a non probability-based way. They usually also send many more surveys per month to their panellists. Both together can generate professional respondents whose quality of answers may be different. Therefore, it makes sense to make a similar comparison for a nonprobability-based online panel (Netquest). Although differences are found, the similarities prevail. Overall, we cannot say that one of the surveys has higher estimates of quality, when defined as the product of reliability and validity, than the other.