KGTN-ens: few-shot image classification with knowledge graph ensembles

Abstract

AbstractWe propose KGTN-ens, a framework extending the recent Knowledge Graph Transfer Network (KGTN) in order to incorporate multiple knowledge graph embeddings at a small cost. There are many real-world scenarios in which the amount of data is severely limited (e.g. health industry, rare anomalies). Prior knowledge can be used to tackle this task. In KGTN, one can use a single knowledge source at once. The purpose of this study is to investigate the possibility of combining multiple knowledge sources. We evaluate it with different embeddings in a few-shot image classification task. Our model is partially trained on $$k \in \{ 1, 2, 5, 10\}$$ k ∈ { 1 , 2 , 5 , 10 } samples. We also construct a new knowledge source – Wikidata embeddings – and evaluate it with KGTN and KGTN-ens. With ResNet50, our approach outperforms KGTN in terms of the top-5 accuracy on the ImageNet-FS dataset for the majority of tested settings. For $$k \in \{ 1, 2, 5, 10\}$$ k ∈ { 1 , 2 , 5 , 10 } respectively, we obtained +0.63/+0.58/+0.43/+0.26 pp. (novel classes) and +0.26/+0.25/+0.32/–0.04 pp. (all classes).