Memory-Efficient Discrete Cosine Transform Domain Weight Modulation Transformer for Arbitrary-Scale Super-Resolution
-
Published:2023-09-18
Issue:18
Volume:11
Page:3954
-
ISSN:2227-7390
-
Container-title:Mathematics
-
language:en
-
Short-container-title:Mathematics
Author:
Kim Min Hyuk1, Yoo Seok Bong1ORCID
Affiliation:
1. Deparment of Artificial Intelligence Convergence, Chonnam National University, Gwangju 61186, Republic of Korea
Abstract
Recently, several arbitrary-scale models have been proposed for single-image super-resolution. Furthermore, the importance of arbitrary-scale single image super-resolution is emphasized for applications such as satellite image processing, HR display, and video-based surveillance. However, the baseline integer-scale model must be retrained to fit the existing network, and the learning speed is slow. This paper proposes a network to solve these problems, processing super-resolution by restoring the high-frequency information lost in the remaining arbitrary-scale while maintaining the baseline integer scale. The proposed network extends an integer-scaled image to an arbitrary-scale target in the discrete cosine transform spectral domain. We also modulate the high-frequency restoration weights of the depthwise multi-head attention to use memory efficiently. Finally, we demonstrate the performance through experiments with existing state-of-the-art models and their flexibility through integration with existing integer-scale models in terms of peak signal-to-noise ratio (PSNR) and similarity index measure (SSIM) scores. This means that the proposed network restores high-resolution (HR) images appropriately by improving the image sharpness of low-resolution (LR) images.
Funder
Industrial Fundamental Technology Development Progra IITP
Subject
General Mathematics,Engineering (miscellaneous),Computer Science (miscellaneous)
Reference37 articles.
1. Zhang, Y., Huang, Y., Wang, K., Qi, G., and Zhu, J. (2023). Single image super-resolution reconstruction with preservation of structure and texture details. Mathematics, 11. 2. Cha, Z., Xu, D., Tang, Y., and Jiang, Z. (2023). Meta-Learning for Zero-Shot Remote Sensing Image Super-Resolution. Mathematics, 11. 3. Dong, C., Loy, C.C., He, K., and Tang, X. (2014, January 6–12). Learning a deep convolutional network for image super-resolution. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. 4. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., and Fu, Y. (2018, January 8–14). Image super-resolution using very deep residual channel attention networks. Proceedings of the European Conference on Computer Vision, Munich, Germany. 5. Cao, J., Wang, Q., Xian, Y., Li, Y., Ni, B., Pi, Z., Zhang, K., Zhang, Y., Timofte, R., and Van Gool, L. (2023, January 20–22). Ciaosr: Continuous implicit attention-in-attention network for arbitrary-scale image super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.
|
|