Image-free single-pixel keypoint detection for privacy preserving human pose estimation

Abstract

Computer vision technology has been applied in various fields such as identification, surveillance, and robot vision. However, computer vision algorithms used for human-related tasks operate on human images, which raises data security and privacy concerns. In this Letter, we propose an image-free human keypoint detection technique using a few coded illuminations and a single-pixel detector. Our proposed method can complete the keypoint detection task at an ultralow sampling rate on a measured one-dimensional sequence without image reconstruction, thus protecting privacy from the data collection stage and preventing the acquisition of detailed visual information from the source. The network is designed to optimize both the illumination patterns and the human keypoint predictor with an encoder–decoder framework. For model training and validation, we used 2000 images from Leeds Sport Dataset and COCO Dataset. By incorporating EfficientNet backbone, the inference time is reduced from 4 s to 0.10 s. In the simulation, the proposed network achieves 91.7% average precision. Our experimental results show an average precision of 88.4% at a remarkably low sampling rate of 0.015. In summary, our proposed method has the advantages of privacy protection and resource efficiency, which can be applied to many monitoring and healthcare tasks, such as clinical monitoring, construction site monitoring, and home service robots.