Knowledge about a Common Source Can Promote Visual — Haptic Integration

Abstract

The brain integrates object information from multiple sensory systems to form a unique representation of our environment. Temporal synchrony and spatial coincidence are important factors for multisensory integration, indicating that the multisensory signals come from a common source. Spatial separations can lead to a decline of visual – haptic integration (Gepshtein et al, 2005 Journal of Vision5 1013–1023). Here we tested whether prior knowledge that two signals arise from the same object can promote integration even when the signals are spatially discrepant. In one condition, participants had direct view of the object they touched. In a second condition, mirrors were used to create a spatial separation between the seen and the felt object. Participants saw the mirror and their hand in the mirror exploring the object and thus knew that they were seeing and touching the same object. To determine the visual — haptic interaction we created a conflict between the seen and the felt shape using an optically distorting lens that made a rectangle look like a square. Participants judged the shape of the probe by selecting a comparison object matching in shape. We found a mutual biasing effect of shape information from vision and touch, independent of whether participants directly looked at the object they touched or whether the seen and the felt object information was spatially separated with the aid of a mirror. This finding suggests that prior knowledge about object identity can promote integration, even when information from vision and touch is provided at spatially discrepant locations.