Affiliation:
1. Mechanical Engineering School, Southeast University, Nanjing 210096, China
2. School of Electronic and Information Engineering, Beihang University, Beijing 100191, China
Abstract
For Synthetic Aperture Radar (SAR) image registration, successive processes following feature extraction are required by both the traditional feature-based method and the deep learning method. Among these processes, the feature matching process—whose time and space complexity are related to the number of feature points extracted from sensed and reference images, as well as the dimension of feature descriptors—proves to be particularly time consuming. Additionally, the successive processes introduce data sharing and memory occupancy issues, requiring an elaborate design to prevent memory leaks. To address these challenges, this paper introduces the OptionEM-based reinforcement learning framework to achieve end-to-end SAR image registration. This framework outputs registered images directly without requiring feature matching and the calculation of the transformation matrix, leading to significant processing time savings. The Transformer architecture is employed to learn image features, while a correlation network is introduced to learn the correlation and transformation matrix between image pairs. Reinforcement learning, as a decision process, can dynamically correct errors, making it more-efficient and -robust compared to supervised learning mechanisms such as deep learning. We present a hierarchical reinforcement learning framework combined with Episodic Memory to mitigate the inherent problem of invalid exploration in generalized reinforcement learning algorithms. This approach effectively combines coarse and fine registration, further enhancing training efficiency. Experiments conducted on three sets of SAR images, acquired by TerraSAR-X and Sentinel-1A, demonstrated that the proposed method’s average runtime is sub-second, achieving subpixel registration accuracy.
Funder
National Natural Science Foundation of China
Subject
General Earth and Planetary Sciences
Reference54 articles.
1. Distinctive image features from scale-invariant keypoints;Lowe;Int. J. Comput. Vis.,2004
2. SAR-SIFT: A SIFT-like algorithm for SAR images;Dellinger;IEEE Trans. Geosci. Remote Sens.,2014
3. Pan, B., Jiao, R., Wang, J., Han, Y., and Hang, H. (2022, January 23–25). SAR image registration based on KECA-SAR-SIFT operator. Proceedings of the 2022 2nd International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI), Nanjing, China.
4. Hossein-Nejad, Z., and Nasri, M. (2022, January 23–24). Image Registration Based on Redundant Keypoint Elimination SARSIFT Algorithm and MROGH Descriptor. Proceedings of the 2022 International Conference on Machine Vision and Image Processing (MVIP), Ahvaz, Iran.
5. Combining optimized SAR-SIFT features and RD model for multisource SAR image registration;Wang;IEEE Trans. Geosci. Remote Sens.,2021