Affiliation:
1. University of Missouri-Kansas City, MO, USA
2. University of Science and Technology of China, He Fei, An Hui, China
3. Tencent America, Palo Alto, CA, USA
Abstract
The block-based coding structure in the hybrid video coding framework inevitably introduces compression artifacts such as blocking, ringing, and so on. To compensate for those artifacts, extensive filtering techniques were proposed in the loop of video codecs, which are capable of boosting the subjective and objective qualities of reconstructed videos. Recently, neural network-based filters were presented with the power of deep learning from a large magnitude of data. Though the coding efficiency has been improved from traditional methods in High-Efficiency Video Coding (HEVC), the rich features and information generated by the compression pipeline have not been fully utilized in the design of neural networks. Therefore, in this article, we propose the Residual-Reconstruction-based Convolutional Neural Network (RRNet) to further improve the coding efficiency to its full extent, where the compression features induced from bitstream in form of prediction residual are fed into the network as an additional input to the reconstructed frame. In essence, the residual signal can provide valuable information about block partitions and can aid reconstruction of edge and texture regions in a picture. Thus, more adaptive parameters can be trained to handle different texture characteristics. The experimental results show that our proposed RRNet approach presents significant BD-rate savings compared to HEVC and the state-of-the-art CNN-based schemes, indicating that residual signal plays a significant role in enhancing video frame reconstruction.
Funder
National Science Foundation (NSF) Center for Big Learning
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Networks and Communications,Hardware and Architecture
Reference58 articles.
1. Fabrice Bellard. 2019. FFmpeg software A complete cross-platform solution to record convert and stream audio and video. Retrieved from http://ffmpeg.org/.
2. Gisle Bjontegaard. 2001. Calculation of average PSNR differences between RD-curves. Document VCEG-M33 Austin Texas.
3. Benjamin Bross Jianle Chen and Shan Liu. 2019. Versatile video coding (draft 4). Document ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 JVET-M1001-v6 Marrakech MA.
4. Variational lossy autoencoder;Chen Xi;arXiv preprint arXiv:1611.02731,2016
Cited by
4 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. A Reconfigurable Framework for Neural Network Based Video In-Loop Filtering;ACM Transactions on Multimedia Computing, Communications, and Applications;2024-03-08
2. A Small-Scale Image U-Net-Based Color Quality Enhancement for Dense Point Cloud;IEEE Transactions on Consumer Electronics;2024-02
3. On Content-Aware Post-Processing: Adapting Statistically Learned Models to Dynamic Content;ACM Transactions on Multimedia Computing, Communications, and Applications;2023-09-18
4. NR-CNN: Nested-Residual Guided CNN In-loop Filtering for Video Coding;ACM Transactions on Multimedia Computing, Communications, and Applications;2022-03-04