Oral and Written Symbol Comprehension Testing: The Benefit of Cognitive Interview Probing

Abstract

Traditionally, symbol comprehension is tested using open-ended, written responses. However, responses are often so brief that they may fail to indicate a participant's true understanding of some symbols. In the present study, several test methods were compared to the standard written method to determine if they produce better symbol comprehension performance. The four alternative methods included: written test with probe questioning after all responses were provided, oral test without probe questioning, oral test with probe questioning after responses for all symbols were provided, and oral test with probe questioning after each partially correct or incorrect response. The probe or follow-up questioning technique is taken from the cognitive interview procedure used in eyewitness identification research to elicit more detailed responses. Participants reported their interpretations of 31 safety symbols in one of the 5 test method conditions. Results showed that the test methods that included follow-up questioning elicited more information from participants, and increased comprehension rates in both oral and written test formats. The results have implications for cost-effective symbol design and evaluation.