DiRaC-I: Identifying Diverse and Rare Training Classes for Zero-Shot Learning

Abstract

Zero-Shot Learning (ZSL) is an extreme form of transfer learning that aims at learning from a few “seen classes” to have an understanding about the “unseen classes” in the wild. Given a dataset in ZSL research, most existing works use a predetermined, disjoint set of seen-unseen classes to evaluate their methods. These seen (training) classes might be sub-optimal for ZSL methods to appreciate the diversity and rarity of an object domain. Inspired by strategies like active learning, it is intuitive that intelligently selecting the training classes can improve ZSL performance. In this work, we propose a framework called Diverse and Rare Class Identifier (DiRaC-I) which, given an attribute-based dataset, can intelligently yield the most suitable “seen classes” for training ZSL models. DiRaC-I has two main goals – constructing a diversified set of seed classes, and using them to initialize a visual-semantic mining algorithm for acquiring the classes capturing both diversity and rarity in the object domain adequately. These classes can then be used as “seen classes” to train ZSL models for image classification. We simulate a real-world scenario where visual samples of novel object classes in the wild are available to neither DiRaC-I nor the ZSL models during training and conducted extensive experiments on two benchmark data sets for zero-shot image classification — CUB and SUN. Our results demonstrate DiRaC-I helps ZSL models to achieve significant classification accuracy improvements – specifically, up to 8% for CUB and up to 5% for SUN dataset. Additionally, while recognizing classes exhibiting rare attributes we also observe a performance boost for ZSL models, which is up to 10% and 7% for CUB and SUN datasets, respectively.