Individual variations in McGurk illusion susceptibility reflect different integration-segregation strategies of audiovisual speech perception

Abstract

AbstractThe McGurk illusion is a widely used indicator of audiovisual speech integration. In this illusion, incongruent visual articulations biases the auditory speech percepts, resulting in illusory percepts that different from both auditory and visual inputs. Despite its widespread use, its validity for measuring audiovisual integration has been questioned due to substantial individual variations. Classical forced fusion theories propose that variations in McGurk illusion susceptibility reflect differences in multisensory integration ability, whereas Bayesian causal inference (BCI) theory proposes that these variations reflect different audiovisual integration-segregation strategies used according to unisensory accuracy and inferred casual structures of the auditory and visual signals. To test these two proposals, we investigated the relationships between variations in McGurk illusion susceptibility and unisensory accuracy across testing time (Experiment 1, N =161), task type (Experiment 2, N = 88), and stimuli (Experiment 3, N = 37). We found stable negative correlations between McGurk illusion susceptibility and unisensory accuracy both at the group level and single participant level. Participants with weak illusion susceptibility had higher unisensory accuracy than participants with strong illusion susceptibility. Conversely, participants with strong illusion susceptibility were more likely to incorrectly categorize the auditory and visual stimuli as illusory percepts. Moreover, participants with similar unisensory accuracy showed similar McGurk illusion susceptibility. Consistent with the BCI theory, our results indicate that individual variations in McGurk illusion susceptibility reflect different integration-segregation strategies used by participants due to variations in unisensory accuracy. Failing to perceive an illusion does not indicate a failure to integrate audiovisual signals; instead, it indicates a successful segregation of incongruent signals. These findings suggest that caution is necessary when generalizing variations in McGurk illusion susceptibility to differences in audiovisual integration ability.