Augmented Reality-Based 3D Human Hands Tracking From Monocular True Images Using Convolutional Neural Network

Abstract

Precise modeling of hand tracking from monocular camera calibration parameters using semantic cues is an active area of research for the researchers due to lack of accuracy and computational overheads. In this context, deep learning-based framework (i.e., convolutional neural network-based human hands tracking in the current camera frame) has become an active research problem. In addition, tracking based on monocular camera needs to be addressed due to updated technology such as Unity3D engine and other related augmented reality plugins. This research aims to track human hands in continuous frame by using the tracked points to draw 3D model of the hands as an overlay. In the proposed methodology, Unity3D environment was used for localizing hand object in augmented reality (AR). Later, convolutional neural network was used to detect hand palm and hand keypoints based on cropped region of interest (ROI). The proposed method achieved accuracy rate of 99.2% where single monocular true images were used for tracking. Experimental validation shows the efficiency of the proposed methodology.