Do we know what we are asking? Individual and group cognitive interviews1

Abstract

Abstract The paper deals with cognitive interview, a method for pre-testing survey questions that is used in pilot testing to develop new measures and/or adapt ones in foreign languages. The aim is to explore the usefulness of the method by looking at two questionnaires measuring anti-Roma prejudice. The first, the Stereotype Content Model (SCM), contains questions that are dominantly used to test two dimensions of social perceptions of various groups: warmth and competence. The second, Interventions for Reducing Prejudice against Stigmatized Minorities (INTERMIN) consists of the items most frequently used in contact research to measure attitudes, social distance, anxiety, trust and behavioural intentions towards outgroups. Two rounds of cognitive interviews were held on both questionnaires to verbally evaluate participants’ understanding and/or interpretation of the draft questions. The first round was attended by university students, while the second round (with improved versions of the questionnaires) was done with high school students, as they are the target group for planned interventions based on the contact paradigm. The paper explains the problems/difficulties the participants had answering some of the questions and our attempts at improving the questionnaires. The problems can be grouped around six issues: The first two deal with the strategies participants used to answer our questions – whom exactly did they have in mind when answering the questionnaires and whose viewpoint did they represent in their answers. The next four problems are around nuances in the formulations of our questions and generally have to do with how the participants interpreted our questions – they concern assumptions that distinct items were logically interconnected, the period of time and locality referred to in our questions, translation and transferability of meanings from one language to another and double negation.