Stereo-measurement of 3D poses of a walking person with a puppet model and low-rank filter

Abstract

To achieve safety and security in public places where many people gather, such as train stations, it is important to recognize people's behavior and to dangerous one. For that purpose, it is necessary to measure a person pose and capture his or her behavior as its changes over time. Conventionally, it is hard to recognize the person itself before capturing the pose, and researchers have been carried out based on the silhouette of the person obtained by performing such as background subtraction. Efficient recognition was difficult when silhouettes were used because they were greatly affected by the observation direction. Since the influence of the observation direction disappears if processing is performed in 3D, some methods using Kinect have been proposed, but in the case of Kinect, the usage environment was limited. Here, OpenPose is applied to the stereo-pair images for getting the two-dimensional coordinates of the joint points. The coordinate output of OpenPose has fluctuation depending on the lighting environment. Assuming that this fluctuation follows a Gaussian distribution, the three-dimensional coordinates will be mixed with a larger noise component following a Cauchy distribution. Since the Cauchy distribution does not have a defined mean or variance, it is impossible to achieve the desired smoothing even with averaging. The method described here provides stable pose data that can withstand subsequent use by applying a low rank filter and a Puppet model in which the distance between joint points is invariant. By updating the model to be used with a more accurate one, it will be possible to extract the walking habits of each person, and it is expected to be applied to gait recognition.