Deep-learning for automated markerless tracking of infants general movements

Abstract

AbstractThe presence of abnormal infant General Movements (GMs) is a strong predictor of progressive neurodevelopmental disorders, including cerebral palsy (CP). Automation of the assessment will overcome scalability barriers that limit its delivery to at-risk individuals. Here, we report a robust markerless pose-estimation scheme, based on advanced deep-learning technology, to track infant movements in consumer mobile device video recordings. Two deep neural network models, namely Efficientnet-b6 and resnet-152, were trained on manually annotated data across twelve anatomical locations (3 per limb) in 12 videos from 6 full-term infants (mean age = 17.33 (SD 2.9) wks, 4 male, 2 female), using the DeepLabCut™ framework. K-fold cross-validation indicates the generalization capability of the deep-nets for GM tracking on out-of-domain data with an overall performance of 95.52% (SD 2.43) from the best performing model (Efficientnet-b6) across all infants (performance range: 84.32–99.24% across all anatomical locations). The paper further introduces an automatic, unsupervised strategy for performance evaluation on extensive out-of-domain recordings through a fusion of likelihoods from a Kalman filter and the deep-net. Findings indicate the possibility of establishing an automated GM tracking platform, as a suitable alternative to, or support for, the current observational protocols for early diagnosis of neurodevelopmental disorders in early infancy.