Convolutional Neural Networks for Indian Sign Language Recognition

Abstract

Sign Language has been a crucial means of com- munication for the deaf and mute communities worldwide since ages. In India alone, 1 percent of the population consists of hard of hearing and mute individuals. Hence, to help support these marginalized communities, it is important to make use of techno-logical advancements such as deep learning, computer vision and neural network technologies to create systems and applications that can not only help create sign language recognition software for the deaf community, but also provide means to educate others about sign languages around the world. In this paper, we present a system that utilizes Convolutional Neural Networks to recognize the alphabets A-Z of the Indian Sign Language(ISL) by accepting the real time hand signs performed by the user as input from the users’ camera feed and then displays the recognized alphabet label as output in the form of text and speech. We created a custom Indian sign language dataset for all 26 alphabets for this experimentation. The extraction of key features was performed using CNN, background removal, hand segmentation and thresholding.