Affiliation:
1. Department of Computer Science and Engineering, Incheon National University (INU), Incheon 22012, Republic of Korea
Abstract
While machine translation for spoken language has advanced significantly, research on sign language translation (SLT) for deaf individuals remains limited. Obtaining annotations, such as gloss, can be expensive and time-consuming. To address these challenges, we propose a new sign language video-processing method for SLT without gloss annotations. Our approach leverages the signer’s skeleton points to identify their movements and help build a robust model resilient to background noise. We also introduce a keypoint normalization process that preserves the signer’s movements while accounting for variations in body length. Furthermore, we propose a stochastic frame selection technique to prioritize frames to minimize video information loss. Based on the attention-based model, our approach demonstrates effectiveness through quantitative experiments on various metrics using German and Korean sign language datasets without glosses.
Funder
Incheon National University
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference49 articles.
1. Santos, A.S., and Portes, A.J.F. (2019). Perceptions of deaf subjects about communication in Primary Health Care. Rev. Lat. Am. Enferm., 27.
2. Recognition of sign language motion images;Tamura;Pattern Recognit.,1988
3. Cormier, K., Fox, N., Woll, B., Zisserman, A., Camgöz, N.C., and Bowden, R. (2019, January 29). Extol: Automatic recognition of british sign language using the bsl corpus. Proceedings of the 6th Workshop on Sign Language Translation and Avatar Technology (SLTAT) 2019, Hamburg, Germany.
4. Camgoz, N.C., Hadfield, S., Koller, O., Ney, H., and Bowden, R. (2018, January 18–23). Neural sign language translation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.
5. Sutton-Spence, R., and Woll, B. (1999). The Linguistics of British Sign Language: An Introduction, Cambridge University Press.
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献