Pixel Self-Attention Guided Real-Time Instance Segmentation for Group Raised Pigs

Abstract

Instance segmentation is crucial to modern agriculture and the management of pig farms. In practical farming environments, challenges arise due to the mutual adhesion, occlusion, and dynamic changes in body posture among pigs, making accurate segmentation of multiple target pigs complex. To address these challenges, we conducted experiments using video data captured from varying angles and non-fixed lenses. We selected 45 pigs aged between 20 and 105 days from eight pens as research subjects. Among these, 1917 images were meticulously labeled, with 959 images designated for the training set, 192 for validation, and 766 for testing. To enhance feature utilization and address limitations in the fusion process between bottom-up and top-down feature maps within the feature pyramid network (FPN) module of the YOLACT model, we propose a pixel self-attention (PSA) module, incorporating joint channel and spatial attention. The PSA module seamlessly integrates into multiple stages of the FPN feature extraction within the YOLACT model. We utilized ResNet50 and ResNet101 as backbone networks and compared performance metrics, including AP0.5, AP0.75, AP0.5-0.95, and AR0.5-0.95, between the YOLACT model with the PSA module and YOLACT models equipped with BAM, CBAM, and SCSE attention modules. Experimental results indicated that the PSA attention module outperforms BAM, CBAM, and SCSE, regardless of the selected backbone network. In particular, when employing ResNet101 as the backbone network, integrating the PSA module yields a 2.7% improvement over no attention, 2.3% over BAM, 2.4% over CBAM, and 2.1% over SCSE across the AP0.5-0.95 metric. We visualized prototype masks within YOLACT to elucidate the model’s mechanism. Furthermore, we visualized the PSA attention to confirm its ability to capture valuable pig-related information. Additionally, we validated the transfer performance of our model on a top-down view dataset, affirming the robustness of the YOLACT model with the PSA module.