An LSTM-Based Prediction Method for Lower Limb Intention Perception by Integrative Analysis of Kinect Visual Signal

Abstract

Recently, computer vision and deep learning technology has been applied in various gait rehabilitation researches. Considering the long short-term memory (LSTM) network has been proved an excellent performance in learn sequence feature representations, we proposed a lower limb joint trajectory prediction method based on LSTM for conducting active rehabilitation on a rehabilitation robotic system. Our approach based on synergy theory exploits that the follow-up lower limb joint trajectory, i.e. limb intention, could be generated by joint angles of the previous swing process of upper limb which were acquired from Kinect platform, an advanced computer vision platform for motion tracking. A customize Kinect-Treadmill data acquisition platform was built for this study. With this platform, data acquisition on ten healthy subjects is processed in four different walking speeds to acquire the joint angles calculated by Kinect visual signals of upper and lower limb swing. Then, the angles of hip and knee in one side which were presented as lower limb intentions are predicted by the fore angles of the elbow and shoulder on the opposite side via a trained LSTM model. The results indicate that the trained LSTM model has a better estimation of predicting the lower limb intentions, and the feasibility of Kinect visual signals has been validated as well.