Affiliation:
1. Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, The Netherlands
2. Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD Nijmegen, The Netherlands
Abstract
In human communication, when the speech is disrupted, the visual channel (e.g. manual gestures) can compensate to ensure successful communication. Whether speech also compensates when the visual channel is disrupted is an open question, and one that significantly bears on the status of the gestural modality. We test whether gesture and speech are dynamically co-adapted to meet communicative needs. To this end, we parametrically reduce visibility during casual conversational interaction and measure the effects on speakers' communicative behaviour using motion tracking and manual annotation for kinematic and acoustic analyses. We found that visual signalling effort was flexibly adapted in response to a decrease in visual quality (especially motion energy, gesture rate, size, velocity and hold-time). Interestingly, speech was also affected: speech intensity increased in response to reduced visual quality (particularly in speech-gesture utterances, but independently of kinematics). Our findings highlight that multi-modal communicative behaviours are flexibly adapted at multiple scales of measurement and question the notion that gesture plays an inferior role to speech.
Reference78 articles.
1. Le signe de l’élévation de la voix;Lombard E;Ann. Mal. Oreille Larynx,1911
2. The Lombard effect
3. Davis C, Kim J, Grauwinkel K, Mixdorff H. 2006 Lombard speech: auditory (A), visual (V) and AV effects. In Proceedings of the third international conference on speech prosody, p. 5. Dresden, Germany: TUD Press.
4. Kim J Davis C Vignali G Hill H. 2005 A visual concomitant of the Lombard reflex. 5.
5. The effect of seeing the interlocutor on auditory and visual speech production in noise
Cited by
4 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献