Multisensory perception reflects individual differences in processing temporal correlations

Abstract

AbstractSensory signals originating from a single event, such as audiovisual speech, are temporally correlated. Correlated signals are known to facilitate multisensory integration and binding. We sought to further elucidate the nature of this relationship, hypothesizing that multisensory perception will vary with the strength of audiovisual correlation. Human participants detected near-threshold amplitude modulations in auditory and/or visual stimuli. During audiovisual trials, the frequency and phase of auditory modulations were varied, producing signals with a range of correlations. After accounting for individual differences which likely reflect relative temporal processing abilities of participants’ auditory and visual systems, we found that multisensory perception varied linearly with strength of correlation. Diffusion modelling confirmed this and revealed that correlation is supplied to the decisional system as sensory evidence. These data implicate correlation as an important cue in audiovisual feature integration and binding and suggest correlational strength as an important factor for flexibility in these processes.