A robust and flexible deep-learning workflow for animal tracking

Abstract

AbstractDevelopments in automated animal tracking software are increasing the efficiency of data collection and improving the standardization of behavioural measurements. There are now several open-source tools for tracking laboratory animals, but often these are only accurate under limited conditions (e.g. uniform lighting and background, uncluttered scenes, unobstructed focal animal). Tracking fish presents a particular challenge for these tools because movement at the water’s surface introduces significant noise. Partial occlusion of the focal animal can also be troublesome, particularly when tracking the whole organism. We conducted a behavioural experiment that required us to track the trajectory of a fish as it swam through a field of obstacles. In addition to measuring the body’s trajectory, we also needed to record the position of the obstacles, and to identify when the fish passed through the ‘virtual gates’ between adjacent obstacles and/or the aquarium wall. We automated data collection by employing a range of computer vision and computational geometry algorithms (e.g. object detection and tracking, optical flow, parallel plane homology mapping, Voronoi tessellation). Our workflow is divided into several discrete steps, and provides a set of modular software building blocks that can be adapted to analyse other experimental designs. A detailed tutorial is provided, together with all the data and code required to reproduce our results.