PriMAT: A robust multi-animal tracking model for primates in the wild

Abstract

AbstractDetection and tracking of animals is an important first step for automated behavioral studies in videos. Nowadays, animal tracking is mostly done using deep learning frameworks based on keypoints, which show remarkable results in lab settings with fixed cameras, backgrounds, and lighting. In contrast, multi-animal tracking in the wild is currently underexplored as it presents numerous challenges such as high background variability, lighting conditions, complex motion, and occlusion.We propose a multi-animal tracking model, PriMAT, for nonhuman primates in the wild. The model learns to detect and track primates and other objects of interest from labeled videos or single images using bounding boxes instead of keypoints. Using bounding boxes significantly facilitates data annotation and allows for more robust results. Our one-stage model is conceptually simple but highly flexible, and we add a classification branch that allows us to train individual identification.To evaluate the performance of our model, we applied it in two case studies with Assamese macaques and redfronted lemurs in the wild. We show that with only a few hundred frames labeled with bounding boxes, we can achieve robust tracking results. Combining these results with the classification branch, our model shows an accuracy of 83% in predicting lemur identities.Our approach presents a promising solution for accurately tracking and identifying animals in the wild using deep learning based bounding box tracking, offering researchers a tool to study animal behavior in their natural habitats. Our code, models, training images, and evaluation video sequences are publicly available (https://github.com/ecker-lab/PriMAT-tracking), facilitating its use for animal behavior analyses and future research in this field.