Zero-Shot Image Classification Method Based on Attention Mechanism and Semantic Information Fusion
Author:
Wang Yaru1ORCID, Feng Lilong1, Song Xiaoke1, Xu Dawei12, Zhai Yongjie1ORCID
Affiliation:
1. Department of Automation, North China Electric Power University, Baoding 071003, China 2. State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
Abstract
The zero-shot image classification (ZSIC) is designed to solve the classification problem when the sample is very small, or the category is missing. A common method is to use attribute or word vectors as a priori category features (auxiliary information) and complete the domain transfer from training of seen classes to recognition of unseen classes by building a mapping between image features and a priori category features. However, feature extraction of the whole image lacks discrimination, and the amount of information of single attribute features or word vector features of categories is insufficient, which makes the matching degree between image features and prior class features not high and affects the accuracy of the ZSIC model. To this end, a spatial attention mechanism is designed, and an image feature extraction module based on this attention mechanism is constructed to screen critical features with discrimination. A semantic information fusion method based on matrix decomposition is proposed, which first decomposes the attribute features and then fuses them with the extracted word vector features of a dataset to achieve information expansion. Through the above two improvement measures, the classification accuracy of the ZSIC model for unseen images is improved. The experimental results on public datasets verify the effect and superiority of the proposed methods.
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference43 articles.
1. Deep learning;Lecun;Nature,2015 2. Research progress of zero-shot learning;Sun;Appl. Intell.,2021 3. Li, L.W., Liu, L., Du, X.H., Wang, X., Zhang, Z., Zhang, J., and Liu, J. (2022). CGUN-2A: Deep Graph Convolutional Network via Contrastive Learning for Large-Scale Zero-Shot Image Classification. Sensors, 22. 4. Palatucci, M., Pomerleau, D., and Hinton, G.E. (2009, January 7–10). Zero-shot learning with semantic output codes. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada. 5. Augmented semantic feature based generative network for generalized zero-shot learning;Li;Neural Netw.,2021
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
|
|