BSRT++: Improving BSRT with Feature Enhancement, Weighted Fusion, and Cyclic Sampling

Abstract

Multi-frame super-resolution (MFSR) generates a super-resolution (SR) image from a burst consisting of multiple low-resolution images. Burst Super-Resolution Transformer (BSRT) is a state-of-the-art deep learning model for MFSR. However, in this study, we show that there is room for further improvement of BSRT in the feature extraction and fusion process. Then, we propose a feature enhancement module (FEM), a cyclic sampling module (CSM), and a feature reweighting module (FRM) and integrate them into BSRT. Finally, we demonstrate that the modules can help recover the high-frequency information well, enhance inter-frame communication, and suppress misaligned features, thus significantly improving the SR performance and producing more visually plausible and pleasant results compared to other MFSR methods, including BSRT. On the SyntheticBurst and RealBurst datasets, the improved BSRT with the modules, dubbed BSRT++, achieved higher PSNR values of 1.15 dB and 1.31 dB than BSRT, respectively.