Beat gestures modulate auditory integration in speech perception

Abstract

In everyday life, people interact with each others through verbal communication but also by spontaneous beat gestures which are a very important part of the paralinguistic context during face-to-face conversations. Nonetheless, their role and neural correlates have been seldom addressed. Here we investigate the time course of beat-speech integration in natural speech perception conditions. We measured event-related potentials to words pronounced with or without an accompanying beat gesture while participants attended to a political speech. When the speaker was on sight, words pronounced with a beat gesture elicited appositive shift in ERPs at early sensory (before 100 ms) and at a later time window coinciding with the auditory component P2. This result remained partially true even when the auditory signal was removed from audiovisual signal. Interestingly, there was no difference with words pronounced without gesture when participants listened to the same speech passage without viewing of the speaker. We conclude that in a naturalistic speech context, beat gestures are integrated with speech early on in time and modulate the sensory/phonological levels of processing. We propose that these results suggest a possible role of beats as a highlighter, helping direct the focus of attention of the listener on important information, rather than adding information per se. Beat gestures would modulate how verbal information is treated.