Comparing the Rate-All-That-Apply and Rate-All-Statements Question Formats across Five Countries

Abstract

Rate All That Apply (RATA) is a derivative of the popularly used Check-All-That-Apply (CATA) question format. For RATA, consumers select all terms or statements that apply from a given list and then continue to rate those selected based on how much they apply. With Rate All Statements (RATING), a widely used standard format for testing, consumers are asked to rate all terms or statements according to how much they apply. Little is known of how the RATA and RATING question formats compare in terms of aspects such as attribute discrimination and sample differentiation. An online survey using either a RATA or RATING question format was conducted in five countries (Brazil, China, India, Spain, and the USA). Each respondent was randomly assigned one of the two question formats (n = 200 per country per format). Motivations for eating items that belong to five food groups (starch-rich, protein-rich, dairy, fruits and vegetables, and desserts) were assessed. More “apply” responses were found for all eating motivation constructs within RATING data than RATA data. Additionally, the standard indices showed that RATING discriminated more among motivations than RATA. Further, the RATING question format showed better discrimination ability among samples for all motivation constructs than RATA within all five countries. Generally, mean scores for motivations were higher when RATA was used, suggesting that consumers who might choose low numbers in the RATING method decide not to check the term in RATA. More investigation into the validity of RATA and RATING data is needed before use of either question format over the other can be recommended.