Affiliation:
1. Bielefeld University, Germany
2. Te Herenga Waka – Victoria University of Wellington, New Zealand
Abstract
This study investigates the synchronization of manual gestures with prosody and information structure using Turkish natural speech data. Prosody has long been linked to gesture as a key driver of gesture–speech synchronization. Gesture has a hierarchical phrasal structure similar to prosody. At the lowest level, gesture has been shown to be synchronized with prosody (e.g., apexes and pitch accents). However, less is known about higher levels. Even less is known about timing relationships with information structure, though this is signaled by prosody and linked to gesture. The present study analyzed phrase synchronization in 3 hr of narrations in Turkish annotated for gesture, prosody, and information structure—topics and foci. The analysis of 515 gesture phrases showed that there was no one-to-one synchronization with intermediate phrases, but their onsets and offsets were synchronized. Moreover, information structural units, topics, and foci were closely synchronized with gesture phrase medial stroke + post-hold combinations (i.e., apical areas). In addition, iconic and metaphoric gestures were more likely to be paired with foci, and deictics with topics. Overall, the results confirm synchronization of gesture and prosody at the phrasal level and provide evidence that gesture shows a direct sensitivity to information structure. These show that speech and gesture production are more connected than assumed in existing production models.
Funder
New Zealand Aid Programme
Subject
Speech and Hearing,Linguistics and Language,Sociology and Political Science,Language and Linguistics,General Medicine
Reference120 articles.
1. Cross-cultural similarities in gestures: The deep relationship between gestures and speech which transcends language barriers
2. Alexanderson S., House D., Beskow J. (2013). Aspects of co-occurring syllables and head nods in spontaneous dialogue. In Ouni S., Berthommier F., Jesse A. (Eds.), Proceedings of 12th international conference on auditory-visual speech processing (pp. 169–172). https://www.isca-speech.org/archive_v0/avsp13/papers/av13_169.pdf
3. Multimodal prominences: Exploring the patterning and usage of focal pitch accents, head beats and eyebrow beats in Swedish television news readings
4. Ambrazaitis G., Svensson Lundmark M., House D. (2015). Head beats and eyebrow movements as a function of phonological prominence levels and word accents in Stockholm Swedish news broadcasts. In Proceedings of the 1st joint conference on facial analysis, animation, and auditory-visual speech processing (p. 42). http://www.diva-portal.org/smash/get/diva2:1306674/FULLTEXT01.pdf
5. Stability of tonal alignment: the case of Greek prenuclear accents