Dactylology Prediction Using Convolution Neural Networks

Abstract

Dactylology is a technique used by individuals who are deaf or heard of hearing to communicate by making signs with their fingers, particularly in manual alphabets. The goal of this project is to create a functional, real-time American Sign Language (ASL) recognition system using vision-based methods through finger spelling gestures and provide real-time text or speech outputs for individuals who are deaf and mute. A convolution neural network (CNN) algorithm has been employed. A major benefit of CNNs is their ability to perform image classification with minimal pre-processing when compared to other algorithms. Unlike other approaches that use manually designed filters, CNNs learn these filters automatically through training. By properly displaying the ASL symbols and ensuring adequate lighting without background noise, the system was able to detect nearly all of the symbols accurately. The proposed methodology achieved an accuracy of 94.8% for the 26 letters of the alphabet.