Transcribing and annotating multimodality

Abstract

This paper deals with the central question of transcribing deaf children’s productions. We present the annotation grid we created on Elan®, explaining in detail how and why the observation of the narrative productions of 6 to 12 year-old deaf children led us to modify the annotation schemes previously available. Deaf children resort to every resource available in both modalities: voice and gesture. Thus, these productions are fundamentally multimodal and bilingual. In order to describe these specific practices, we propose considering verbal and non-verbal, vocal and gestural, materials as parts of one integrated production. A linguistic-centered transcription is not efficient in describing such bimodal productions, since describing bimodal utterances implies taking into account the ‘communicative desire’ (‘vouloir-dire’) of the children. For this reason, both the question of the transcription unit and the issue of the complexity of semiotic interactions in bimodal utterances need to be reconsidered.