Rapid Semantic Annotation as an Evaluative Tool for Warning Symbol Design

Abstract

A key step in the stereotype production method, and in other methods of warning symbol development, is the evaluation of symbol sketches produced by naïve design participants. These sketches provide valuable insight into the expectations of potential warning symbol users about the content and design of the symbol. Until recently, the symbol development process depended heavily on designers’ expertise to judge the intent of a participant’s sketch and its similarities to other participants’ drawings. However, mathematical and computational techniques have been introduced that assist symbol designers by clustering similar sketches by their graphical attributes and evolving new designs from prior sketches based on the most preferred symbolic components. Semantic annotation is used to attach standardized descriptive terms to the sketches which the clustering process uses to discriminate between the otherwise very subjective symbol drawings. Unfortunately, semantic annotation can be a lengthy process of annotators studying the nuances of each sketch. This is incongruent with the real-world usage of warning symbols where an individual’s attention is fixated on a symbol for only a few seconds during which time the message must be received. The process of Rapid Semantic Annotation is proposed to better match the field experience of symbol users to the design process and to reduce the designers’ time spent on semantic annotation. Symbol sketches for two warning referents (“hot exhaust” and “do not touch with wet hands”) were taken from a previous study involving semantic annotation by an expert panel. Thirty sketches were randomly chosen from a larger pool for each referent and presented independently to 6 participants. Participants were given only 5 seconds to view the sketch before recording their semantic annotations in the form of textual terms (e.g. “person” or “flame”). A list of these “attributes” was aggregated from all participants for each of the 30 sketches in a referent, and an annotation matrix was created containing the count (0-6) of participants who annotated each attribute for each sketch. In total, 54 attributes were annotated for the “Hot Exhaust” referent (a 30 x 54 matrix) and 37 attributes were annotated for the “Wet Hands” referent (a 30 x 37 matrix). Weka Simple K-Means clustering was performed on the matrices, and the clustering results are compared to the results produced from the expert panel semantic annotation in the previous study.