Affiliation:
1. School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu 965-8580, Japan
Abstract
Japanese Sign Language (JSL) is vital for communication in Japan’s deaf and hard-of-hearing community. But probably because of the large number of patterns, 46 types, there is a mixture of static and dynamic, and the dynamic ones have been excluded in most studies. Few researchers have been working to develop a dynamic JSL alphabet, and their performance accuracy is unsatisfactory. We proposed a dynamic JSL recognition system using effective feature extraction and feature selection approaches to overcome the challenges. In the procedure, we follow the hand pose estimation, effective feature extraction, and machine learning techniques. We collected a video dataset capturing JSL gestures through standard RGB cameras and employed MediaPipe for hand pose estimation. Four types of features were proposed. The significance of these features is that the same feature generation method can be used regardless of the number of frames or whether the features are dynamic or static. We employed a Random forest (RF) based feature selection approach to select the potential feature. Finally, we fed the reduced features into the kernels-based Support Vector Machine (SVM) algorithm classification. Evaluations conducted on our proprietary newly created dynamic Japanese sign language alphabet dataset and LSA64 dynamic dataset yielded recognition accuracies of 97.20% and 98.40%, respectively. This innovative approach not only addresses the complexities of JSL but also holds the potential to bridge communication gaps, offering effective communication for the deaf and hard-of-hearing, and has broader implications for sign language recognition systems globally.
Funder
Competitive Research Fund of The University of Aizu, Japan
Reference40 articles.
1. Japan, C.O. (2023, June 08). White Paper on Persons with Disabilities 2023. Available online: https://nanbyo.jp/2023/09/12/whitepaper_disabilities/.
2. Kobayashi, H., Ishikawa, T., and Watanabe, H. (2019, January 10–13). Classification of Japanese Signed Character with Pose Estimation and Machine Learning. Proceedings of the IEICE General Conference on Information and Systems, Hiroshima, Japan.
3. Ito, S.i., Ito, M., and Fukumi, M. (2019, January 5–8). A Method of Classifying Japanese Sign Language using Gathered Image Generation and Convolutional Neural Networks. Proceedings of the 2019 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), Fukuoka, Japan.
4. Ministry of Health, Labour and Welfare of Japan (2023, June 08). Survey on Difficulties in Living. Online. Available online: https://www.mhlw.go.jp/toukei/list/seikatsu_chousa_h28.html:.
5. Sign language service with IT for the Deaf people in Japan “Tech for the Deaf”;Ohki;J. Inf. Process. Manag.,2014
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Techniques for Generating Sign Language a Comprehensive Review;Journal of The Institution of Engineers (India): Series B;2024-07-13