AFIRE: Adaptive FusionNet for Illumination-Robust Feature Extraction in Heterogeneous Imaging Environments

Abstract

Abstract The fusion of infrared and visible images aims to synthesize a fused image that incorporates richer information by leveraging the distinct characteristics of each modality. However, the disparate quality of input images in terms of infrared and visible light significantly impacts fusion performance. Addressing this issue, we propose a deep adaptive fusion method in this paper, termed Adaptive FusionNet for Illumination-Robust Feature Extraction (AFIRE), which involves interactive processing of two input features and dynamically adjusts fusion weights under varying illumination conditions. Specifically, we introduce a novel interactive extraction structure during the feature extraction stage for both infrared and visible light, enabling the capture of more complementary information. Additionally, we design a Deep Adaptive Fusion module to assess the quality of input features and perform weighted fusion through a channel attention mechanism. Finally, a new loss function is formulated by incorporating the entropy and median of input images to guide the training of the fusion network. Extensive experiments demonstrate that AFIRE outperforms state-of-the-art methods in preserving pixel intensity distribution and texture details. Source code is available in GitHub https://github.com/ISCLab-Bistu/AFIRE.