Robot Assist Sign Language Recognition for Hearing Impaired Persons Using Deep Learning

Abstract

The adoption of Sign Language Communication (SLC) systems has become more significant in closing the interaction between the deaf society and the world of hearing people. In this study, researchers aim to contribute to this field by developing a system that helps sign language users communicate through BERT approaches based on deep learning frameworks as well as NLP. Accurate recognition of sign language is critical for SLC systems to work well. Deep learning models are effective in identifying sign language with high accuracy. This study aims to determine the most suitable DL model for identifying sign language and assess the impact of incorporating Natural Language Processing (NLP) techniques in generating frequent and accurate responses in SLC systems. The NLP model will be developed as an optimum return mechanism to generate frequent responses. This research includes testing three different deep learning models: MLP, CNN, and RestNet50v2 to recognize sign language gestures. Restnet50v2 outscored the other two approaches with a 0.97% perfection. As said earlier, the system also generates automated responses using the NLP BERT model, with an overall accuracy of 0.8% and a BLEU score of 0.83%. This method has a way to enhance interaction among the deaf community via the use of technology, opening new avenues for developing intelligent chatbots that can better understand nonverbal communication. Further research can be done to expand its functionality to recognize a broader range of sign language gestures and improve the user interface. Overall, this study demonstrates how technology can enhance the ways of people with deafness or hearing loss by addressing communication barriers.