Crossmodal matching

Abstract

Multimodal interfaces which distribute information across vision, audition, and touch have been demonstrated to improve performance in various complex domains. However, many multimodal studies to- date fail to conduct crossmodal matching, a critical step to ensure cues across different sensory channels are perceived to be of equal intensity. The present study compared two different methods of crossmodal matching based on previous work conducted by Stevens – the methods of bracketing and adjustment. Each participant completed the crossmodal matching task using two different interfaces using the method of bracketing or adjustment for all modality combinations across vision, audition, and touch. The results showed a significant effect of interface type and subject variability depending on the modality used as a reference. Overall, the findings show the viability of the new method, but also support the need of a reliable crossmodal matching technique that reduces within-subject variability.