Affiliation:
1. School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China
Abstract
Hyperspectral videos (HSVs) can record more adequate detail clues than other videos, which is especially beneficial in cases of abundant spectral information. Although traditional methods based on correlation filters (CFs) employed to explore spectral information locally achieve promising results, their performances are limited by ignoring global information. In this paper, a joint spectral–spatial information method, named spectral–spatial transformer-based feature fusion tracker (SSTFT), is proposed for hyperspectral video tracking, which is capable of utilizing spectral–spatial features and considering global interactions. Specifically, the feature extraction module employs two parallel branches to extract multiple-level coarse-grained and fine-grained spectral–spatial features, which are fused with adaptive weights. The extracted features are further fused with the context fusion module based on a transformer with the hyperspectral self-attention (HSA) and hyperspectral cross-attention (HCA), which are designed to capture the self-context feature interaction and the cross-context feature interaction, respectively. Furthermore, an adaptive dynamic template updating strategy is used to update the template bounding box based on the prediction score. The extensive experimental results on benchmark hyperspectral video tracking datasets demonstrated that the proposed SSTFT outperforms the state-of-the-art methods in both precision and speed.
Funder
National Natural Science Foundation of China
Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University
Subject
General Earth and Planetary Sciences
Reference60 articles.
1. Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., and Lu, H. (2021, January 19–25). Transformer Tracking. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Online.
2. Deep visual tracking: Review and experimental comparison;Li;Pattern Recognit.,2018
3. A survey on moving object detection and tracking in video surveillance system;Joshi;Int. J. Soft Comput. Eng.,2012
4. Tracking developments in artificial intelligence research: Constructing and applying a new search strategy;Liu;Scientometrics,2021
5. Multi-vehicle tracking using microscopic traffic models;Song;IEEE Trans. Intell. Transp. Syst.,2018
Cited by
3 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献