A hull reconstruction–reprojection method for pose estimation of free-flying fruit flies

Abstract

ABSTRACT Understanding the mechanisms of insect flight requires high-quality data of free-flight kinematics, e.g. for comparative studies or genetic screens. Although recent improvements in high-speed videography allow us to acquire large amounts of free-flight data, a significant bottleneck is automatically extracting accurate body and wing kinematics. Here, we present an experimental system and a hull reconstruction–reprojection algorithm for measuring the flight kinematics of fruit flies. The experimental system can automatically record hundreds of flight events per day. Our algorithm resolves a significant portion of the occlusions in this system by a reconstruction–reprojection scheme that integrates information from all cameras. Wing and body kinematics, including wing deformation, are then extracted from the hulls of the wing boundaries and body. This model-free method is fully automatic, accurate and open source, and can be readily adjusted for different camera configurations or insect species.