Calibrating self-reported church attendance questions in online surveys. Experimental evidence from the Italian context

Abstract

Despite widespread use in survey research, the accuracy and validity of self-reported church attendance questions have often been debated. Since the seminal article by Hadaway et al. (1993), that this indicator leads to an overestimation of the number of regular churchgoers has entered common knowledge. However, no systematic work to improve the understanding and command of the measurement instrument has been carried out. This contribution analyses the effect of different formulations of the self-reported church attendance question in online questionnaires, by means of survey experiments on a sample of Italian Catholics. In particular the most common ‘how often’ version of the question is compared to an alternative version asking how many times respondents went to church in the last month. The experimental results show that, despite criticism, the ‘how often’ version remains the best option for obtaining information on individual religious practice in survey research. This version is robust to changes in the formulation of answer categories and produces more informative results for respondents with low attendance. Finally, the study supplies evidence consistent with the growing body of literature that underlines the role of religious identity and self-conception in answering questions on church attendance.