Motion recognition using deep convolutional neural network for Kinect-based NAO teleoperation

Abstract

AbstractThe capabilities of teleoperated robots can be enhanced with the ability to recognise and reproduce human-like behaviour. The proposed framework presents motion recognition for a Kinect-based NAO teleoperation. It allows the NAO robot to recognise the human motions and act as a human motion imitator. A stable whole-body imitation is still a challenging issue because of the difficulty in dynamic balancing of centre of mass (CoM). In this paper, a novel adaptive balancing technique for NAO (ABTN) is proposed to control the whole body in single as well as double supporting phases. It targets dynamic balancing of the humanoid robot by solving forward kinematics and applying a weighted average of mass with the CoMs of individual links with respect to the previous joint frames, which provides us with the dynamic CoM of the whole body. Our novel approach uses this dynamic CoM and calculates joint angles using proposed pitch and roll control algorithm to keep the dynamic CoM inside the stable region. Additionally, the NAO robot is capable of recognising human motions using the proposed 7-layer one-dimensional convolutional neural network (1D-CNN). To solve the problem of variable length of time sequences, Zero padding is introduced with 1D-CNN. It attains a recognition accuracy of 95% as compared to the hidden Markov model and neural network. The experimental results demonstrate that the developed teleoperation framework is robust and serves as potential support for the development and application of teleoperated robots.