VISUAL SPEECH RECOGNITION USING DYNAMIC FEATURES AND SUPPORT VECTOR MACHINES

Abstract

This paper presents a vision based technique to identify the unspoken phones using a small camera that is located on the headset of the speaker. The system is based on temporal integration of the video data to generate motion history image (MHI). The paper proposes the use of global features to classify the MHI and compares the use of image moments with Discrete Cosine Transform (DCT). A comparison between Zernike moments (ZM) with DCT indicates that while the accuracy of classification for both techniques is very comparable (96% for ZM and 94% for DCT) when there is no relative motion between the camera and the mouth, ZM is resilient to rotation of the camera and continues to gives good results despite rotation but DCT is sensitive to rotation. Based on the accuracy of the system and its resilience to movement artefacts such as rotation, the authors propose the use of such a system for human computer interface. Such a system could be invaluable when it is important to communicate without making a sound, such as giving passwords when in an open office or in public spaces.