Intelligently Counting Agricultural Pests by Integrating SAM with FamNet

Abstract

The utilization of the large pretrained model (LPM) based on Transformer has emerged as a prominent research area in various fields, owing to its robust computational capabilities. However, there remains a need to explore how LPM can be effectively employed in the agricultural domain. This research aims to enhance agricultural pest detection with limited samples by leveraging the strong generalization performance of the LPM. Through extensive research, this study has revealed that in tasks involving the counting of a small number of samples, complex agricultural scenes with varying lighting and environmental conditions can significantly impede the accuracy of pest counting. Consequently, accurately counting pests in diverse lighting and environmental conditions with limited samples remains a challenging task. To address this issue, the present research suggests a unique approach that integrates the outstanding performance of the segment anything model in class-agnostic segmentation with the counting network. Moreover, by intelligently utilizing a straightforward TopK matching algorithm to propagate accurate labels, and drawing inspiration from the GPT model while incorporating the forgetting mechanism, a more robust model can be achieved. This approach transforms the problem of matching instances in different scenarios into a problem of matching similar instances within a single image. Experimental results demonstrate that our method enhances the accuracy of the FamNet baseline model by 69.17% on this dataset. Exploring the synergy between large models and agricultural scenes warrants further discussion and consideration.