Multimodal image translation algorithm based on Singular Squeeze-and-Excitation Network

Abstract

Image-to-image translation methods have evolved from only considering image-level information to pixel-level and instance-level information. However, with the feature-level constraint, when channel attention (SEnet) extracts content features, its scaling degree does not add effective constraints. To address this difficulty, the multimodal image translation algorithm based on Singular Squeeze-and-Excitation Network (MUNSSE) is proposed by combining deep learning methods and traditional mechanism methods. This method used the mean idea of SVD features to help SEnet ease the degree of scaling. Specifically, SEnet used SVD to extract features to improve the Excitation operation, which helps the network to obtain new channel attention weights and form the attention feature maps.Then the the image content features are completed by convolutional features maps and attention feature maps. Finally, the content features and style features extracted by the style network are combined to obtain the new style images. Through ablation experiments, we found that the SVD parameter is 128, and the image translated by the network is optimal. According to the FID image diversity index, MUNSSE is superior to the method proposed at this stage for the diversity of generated images.