Active ordinal classification by querying relative information

Abstract

Collecting and learning with auxiliary information is a way to further reduce the labeling cost of active learning. This paper studies the problem of active learning for ordinal classification by querying low-cost relative information (instance-pair relation information) through pairwise queries. Two challenges in this study that arise are how to train an ordinal classifier with absolute information (labeled data) and relative information simultaneously and how to select appropriate query pairs for querying. To solve the first problem, we convert the absolute and relative information into the class interval-labeled training instances form by introducing a class interval concept and two reasoning rules. Then, we design a new ordinal classification model for learning with the class interval-labeled training instances. For query pair selection, we specify that each query pair consists of an unlabeled instance and a labeled instance. The unlabeled instance is selected by a margin-based critical instance selection method, and the corresponding labeled instance is selected based on an expected cost minimization strategy. Extensive experiments on twelve public datasets validate that the proposed method is superior to the state-of-the-art methods.