Exploring crossmodal perceptual enhancement and integration in a sequence-reproducing task with cognitive priming

Abstract

AbstractCrossmodal correspondence, a perceptual phenomenon which has been extensively studied in cognitive science, has been shown to play a critical role in people’s information processing performance. However, the evidence has been collected mostly based on strictly-controlled stimuli and displayed in a noise-free environment. In real-world interaction scenarios, background noise may blur crossmodal effects that designers intend to leverage. More seriously, it may induce additional crossmodal effects, which can be mutually exclusive to the intended one, leading to unexpected distractions from the task at hand. In this paper, we report two experiments designed to tackle these problems with cognitive priming techniques. The first experiment examined how to enhance the perception of specific crossmodal stimuli, namely pitch–brightness and pitch–elevation stimuli. The second experiment investigated how people perceive and respond to crossmodal stimuli that were mutually exclusive. Results showed that first, people’s crossmodal perception was affected by cognitive priming, though the effect varies according to the combination of crossmodal stimuli and the types of priming material. Second, when two crossmodal stimuli are mutually exclusive, priming on only the dominant one (Pitch–elevation) lead to improved performance. These results can help inform future design of multisensory systems by presenting details of how to enhance crossmodal information with cognitive priming.