The return of non-probability sample: the electoral polls at the time of internet and social media

Abstract

AbstractFor the past 80 years survey researchers have used a probability sampling framework. Probability samples have a well-defined set of quality criteria that have been organized around the concept of Total Survey Error (TSE). Non-probability samples do not fit within this framework very well and some possible alternatives to TSE are explored. In recent years, electoral polls have undergone changes as a result of the dispersion of public opinion due mostly, but not only, to the development of the web and social media. From a methodological point of view, the main changes concerned sampling and data collection techniques. The aim of the article is to provide a critical contribution to the methodological debate on electoral polls with particular attention to the samples used which appear to be more similar to non-probability samples than to the traditional probability samples used for many decades in electoral polls. We will explore several new approaches that attempt to make inference possible even when a survey sample does not match the classic probability sample. We will also discuss a set of post hoc adjustments that have been suggested as ways to reduce the bias in estimates from non-probability samples; these adjustments use auxiliary data in an effort to deal with selection and other biases. Propensity score adjustment is the most well know of these techniques. The empirical section of the article analyzes a database of 1793 electoral polls conducted in Italy from January 2017 to July 2023.