A Comparison of the Percentage of “Yes” (Agree) Responses and Importance of Attributes (Constructs) Determined Using Check-All-That-Apply and Check-All-Statements (Yes/No) Question Formats in Five Countries

Abstract

Check All That Apply (CATA) has become a popular type of questionnaire response in sensory/consumer research in recent years. However, some authors have pointed out potential problems with the method. An online survey using either a Check-All-That-Apply (CATA) or Check-All-Statements (CAS) format for questions was conducted to provide a deeper understanding of the response data using the two question formats. With CATA, respondents select all terms or statements that apply from a given list, while, with CAS, respondents must respond (e.g., yes/no or agree/disagree) to each term or statement to show that it applies or does not apply. Respondents from five countries (Brazil, China, India, Spain, and the USA) were randomly assigned one of the two question formats (N = 200 per country per method). Motivations for eating items that belong to five food groups (starchy, protein, dairy, fruits, and desserts) were assessed. Results showed that CAS had higher percentages of “agree” responses than CATA. Also, the response ratio of CAS and CATA data was different, suggesting that interpretations of the data from each response type would also be different. Respondents in the USA, China, and Spain took longer to complete the CAS questionnaire, while respondents in Brazil and India had similar time durations for the two question formats. Overall, the CATA format was liked slightly more than the CAS format and fewer respondents dropped out of the survey when using the CATA response type. These findings suggest that the CATA format is quick and relatively easy for consumers to complete. However, it provokes fewer “apply” responses, which some psychologists suggest underestimates applicable terms or statements and CATA provides a different interpretation of data than the CAS format that requires consumers to respond to each term or statement. Further, CAS may overestimate the applicable terms. Consumer insights collected using CATA and CAS can lead to different decisions due to differences in data interpretation by researchers (e.g., marketers, nutritionists, product developers, and sensory scientists). More investigation is needed for the CATA and CAS question formats.