Full-Reference Image Quality Assessment Using Self-Attention and Multiscale Features
-
Published:2023-12-29
Issue:
Volume:
Page:
-
ISSN:0218-1266
-
Container-title:Journal of Circuits, Systems and Computers
-
language:en
-
Short-container-title:J CIRCUIT SYST COMP
Author:
Li Yutong1ORCID,
Liao Xiaofeng1ORCID,
Zhou Mingliang1ORCID,
Ji Cheng2ORCID,
Wei Xuekai1ORCID,
Yue Hong3ORCID
Affiliation:
1. College of Computer Science, Chongqing University, Chongqing 400044, P. R. China
2. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, P. R. China
3. CICT Connected and Intelligent Technologies Co. Ltd, Chongqing 400044, P. R. China
Abstract
In this paper, we construct a deep framework for full-reference image quality assessment (FR-IQA) by combining convolution and self-attention features; this approach effectively uses multiscale features to mimic the image evaluation in human eyes. We achieve the integration of information from local to global. First, convolutional neural network (CNN) and encoder of Swin transformer are used to extract multiscale paired features. Second, for each type of features, we fuse them after converting them to a fixed number of channels. For the fused features, we use the square of the difference between two pixel values at the corresponding channel positions to represent the features of the distortion degree. Then, we introduce a recurrent neural network (RNN) to capture the global features. Finally, for the two types of features, we use the full connection (FC) layer to regress two scores and then add the weights to compute the ultimate perceived score. By training and testing on publicly available FR-IQA datasets, experimental results further validate the superiority of our approach.
Funder
NSFC
National Science Youth Fund of Jiangsu Province
National Key R&D Program of China
National Natural Science Foundation of China
Key Projects of Basic Strengthening Plan
Chongqing Talent
Joint Equipment Pre Research and Key Fund Project of the Ministry of Education
Natural Science Foundation of Chongqing, China
Human Resources and Social Security Bureau Project of Chongqing
Guangdong Oppo Mobile Telecommunications Corporation Ltd.
Publisher
World Scientific Pub Co Pte Ltd
Subject
Electrical and Electronic Engineering,Hardware and Architecture,Electrical and Electronic Engineering,Hardware and Architecture
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献