Efficient YOLO-Based Deep Learning Model for Arabic Sign Language Recognition

Abstract

Verbal communication is the dominant form of self-expression and interpersonal communication. Speech is a considerable obstacle for individuals with disabilities, including those who are deaf, hard of hearing, mute, and nonverbal. Sign language is a complex system of gestures and visual signs facilitating individual communication. With the help of artificial intelligence, the hearing and the deaf can communicate more easily. Automatic detection and recognition of sign language is a complex and challenging task in computer vision and machine learning. This paper proposes a novel technique using deep learning to recognize the Arabic Sign Language (ArSL) accurately. The proposed method relies on advanced attention mechanisms and convolutional neural network architecture integrated with a robust You Only Look Once (YOLO) object detection model that improves the detection and recognition rate of the proposed technique. In our proposed method, we integrate the self-attention block, channel attention module, spatial attention module, and cross-convolution module into feature processing for accurate detection. The recognition accuracy of our method is significantly improved, with a higher detection rate of 99%. The methodology outperformed conventional methods, achieving a precision rate of 0.9 and a mean average precision (mAP) of 0.9909 at an intersection over union (IoU) of 0.5. From IoU thresholds of 0.5 to 0.95, the mAP continuously remains high, indicating its effectiveness in accurately identifying signs at different precision levels. The results show the model’s robustness in accurately detecting and classifying complex multiple ArSL signs. The results show the robustness and efficacy of the proposed model.