Touchless Heart Rate Monitoring from an Unmanned Aerial Vehicle Using Videoplethysmography

Abstract

Motivation: The advancement of preventive medicine and, subsequently, telemedicine drives the need for noninvasive and remote measurements in patients’ natural environments. Heart rate (HR) measurements are particularly promising and extensively researched due to their quick assessment and comprehensive representation of patients’ conditions. However, in scenarios such as endurance training or emergencies, where HR measurement was not anticipated and direct access to victims is limited, no method enables obtaining HR results that are suitable even for triage. Methods: This paper presents the possibility of remotely measuring of human HR from a series of in-flight videos using videoplethysmography (VPG) along with skin detection, human pose estimation and image stabilization methods. An unmanned aerial vehicle (UAV) equipped with a camera captured ten segments of video footage featuring volunteers engaged in free walking and running activities in natural sunlight. The human pose was determined using the OpenPose algorithm, and subsequently, skin areas on the face and forearms were identified and tracked in consecutive frames. Ultimately, HR was estimated using several VPG methods: the green channel (G), green-red difference (GR), excess green (ExG), independent component analysis (ICA), and a plane orthogonal to the skin (POS). Results: When compared to simultaneous readings from a reference ECG-based wearable recorder, the root-mean-squared error ranged from 17.7 (G) to 27.7 (POS), with errors of less than 3.5 bpm achieved for the G and GR methods. Conclusions: These results demonstrate the acceptable accuracy of touchless human pulse measurement with the accompanying UAV-mounted camera. The method bridges the gap between HR-transmitting wearables and emergency HR recorders, and it has the potential to be advantageous in training or rescue scenarios in mountain, water, disaster, or battlefield settings.