Interframe Saliency Transformer and Lightweight Multidimensional Attention Network for Real-Time Unmanned Aerial Vehicle Tracking

Abstract

UAV visual-object-tracking technology based on Siamese neural networks has great scientific research and practical application value, and it is widely used in geological surveying, reconnaissance monitoring, and environmental monitoring. Due to the limited onboard computational resources and complex real-world environments of drones, most of the existing tracking systems based on Siamese neural networks struggle to combine excellent performance with high efficiency. Therefore, the key issue is to study how to improve the accuracy of target tracking under the challenges of real-time performance and the above factors. In response to this problem, this paper proposes a real-time UAV tracking system based on interframe saliency transformer and lightweight multidimensional attention network (SiamITL). Specifically, interframe saliency transformer is used to continuously perceive spatial and temporal information, making the network more closely related to the essence of the tracking task. Additionally, a lightweight multidimensional attention network is used to better capture changes in both target appearance and background information, improving the ability of the tracker to distinguish between the target and background. SiamITL is effective and efficient: extensive comparative experiments and ablation experiments have been conducted on multiple aerial tracking benchmarks, demonstrating that our algorithm can achieve more robust feature representation and more accurate target state estimation. Among them, SiamITL achieved success and accuracy rates of 0.625 and 0.818 in the UAV123 benchmark, respectively, demonstrating a certain level of leadership in this field. Furthermore, SiamITL demonstrates the potential for real-time operation on the embedded platform Xavier, highlighting its potential for practical application in real-world scenarios.