Gesture identification in testing mode in the integrated sign language training system

Abstract

Abstract The structure of an integral sign language teaching system based on a 3D computer character, which allows demonstrating fingerprints and words of a sign language using an avatar, is considered. The system provides the test mode when the user must demonstrate the word or fingerprint indicated by the system. The user feedback with the system is carried out using a standard webcam. The problems of gesture localization and identification in user testing mode are considered. The features of this mode are determined during the system operation. The algorithm for quick identification of human hand gestures is proposed. The method based on the signs, which are two-dimensional projections, is used for gesture identification. The projection recognition features are considered, the similarity metrics are analyzed, and three stages for match decision are identified. The experimental results show the stable operation of this algorithm in various lighting conditions and at the place where the gesture was shown.