Author:
Nguyen Thi Bich Diep,Phung Trung Nghia,Vu Tat Thang
Abstract
Sign languages are independent languages of deaf communities. The translation from normal languages (i.e., Vietnamese Language - VL) as long as other sign languages to Vietnamese sign language (VSL) is a meaningful task that breaks down communication barriers and improves the quality of life for the deaf community. In this paper, we experimented with and proposed several methods for building and improving models for the VL to VSL translation task. We presented a data augmentation method to improve the performance of our neural machine translation models. Using an initial dataset of 10k bilingual sentence pairs, we were able to obtain a new dataset of 60k sentence pairs with a perplexity score no more than 1.5 times that of the original dataset. Experiments on the original dataset showed that rule-based models achieved the highest BLEU score of 68.02 among the translation models. However, with the augmented dataset, the Transformer model achieved the best performance with a BLEU score of 89.23, which is significantly better than that of other conventional approach methods.
Publisher
Publishing House for Science and Technology, Vietnam Academy of Science and Technology (Publications)
Subject
Industrial and Manufacturing Engineering,Environmental Engineering
Reference15 articles.
1. . Sandler W., Lillo-Martin. Sign Language and Linguistic Universals. J. Linguist. page 738-742, 2006.
2. Achraf Othman and Mohamed Jemni, Statistical Sign Language Machine Translation from Englishwritten text to American Sign Language Gloss, IJCSI International Journal of Computer Science Issues, Vol. 8, Issue 5, No 3, 2011.
3. Stephen Cox, Michael Lincoln, Judy Tryggvason, Melanie Nakisa, Mark Wells, Marcus Tutt, “Tessa, a system to aid communication with deaf people”, Proceedings of the fifth international ACM conference on Assistive technologies, 2002.
4. J. A. Bangham, S. J. Cox, R. Elliot, J. R. W. Glauert, I. Marshall, S. Rankov, and M. Wells, “Virtual signing: Capture, animation, storage and transmission – An overview of the ViSiCAST project” , IEEE Seminar on Speech and language processing for disabled and elderly people, 2000.
5. Angus Grieve-Smith, SignSynth: A Sign Language Synthesis Application Using Web3D and Perl, Conference: Revised Papers from the International Gesture Workshop on Gesture and Sign Languages in Human-Computer Interaction, 2002.