Abstract
AbstractThis paper presents a novel approach to video super-resolution (VSR) by focusing on the selection of input frames, a process critical to VSR. VSR methods typically rely on deep learning techniques, those that are able to learn features from a large dataset of low-resolution (LR) and corresponding high-resolution (HR) videos and generate high-quality HR frames from any new LR input frames using the learned features. However, these methods often use as input the immediate neighbouring frames to a given target frame without considering the importance and dynamics of the frames across the temporal dimension of a video. This work aims to address the limitations of the conventional sliding-window mechanisms by developing input frame selection algorithms. By dynamically selecting the most representative neighbouring frames based on content-aware selection measures, our proposed algorithms enable VSR models to extract more informative and accurate features that are better aligned with the target frame, leading to improved performance and higher-quality HR frames. Through an empirical study, we demonstrate that the proposed dynamic content-aware selection mechanism improves super-resolution results without any additional architectural overhead, offering a counter-intuitive yet effective alternative to the long-established trend of increasing architectural complexity to improve VSR results.
Publisher
Springer Science and Business Media LLC
Reference44 articles.
1. Wang Z, Chen J, Hoi SC. Deep learning for image super-resolution: A survey. IEEE transactions on pattern analysis and machine intelligence; 2020.
2. Arefin MR, Michalski V, St-Charles P-L, Kalaitzis A, Kim S, Kahou SE, Bengio Y. Multi-image super-resolution for remote sensing using deep recurrent networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020;pp. 206–207.
3. Liang M, Du J, Li L, Xue Z, Wang X, Kou F, Wang X. Video super-resolution reconstruction based on deep learning and spatio-temporal feature self-similarity. IEEE Transactions on Knowledge and Data Engineering. 2020;1–1. https://doi.org/10.1109/TKDE.2020.3034261.
4. Liu Z-S, Siu W-C, Chan Y-L. Efficient video super-resolution via hierarchical temporal residual networks. IEEE Access. 2021;9:106049–64. https://doi.org/10.1109/ACCESS.2021.3098326.
5. Liu H, Ruan Z, Zhao P, Dong C, Shang F, Liu Y, Yang L. Video super resolution based on deep learning: A comprehensive survey. arXiv preprint arXiv:2007.12928 2020.
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. A Survey of Deep Learning Video Super-Resolution;IEEE Transactions on Emerging Topics in Computational Intelligence;2024-08