SWRM: Similarity Window Reweighting and Margin for Long-Tailed Recognition

Abstract

Real-world data usually obeys a long-tailed distribution, where a few classes have higher number of samples compared to the other classes. Recent studies have been proposed to alleviate the extreme data imbalance from different perspectives. In this article, we experimentally find that due to the easily confusing visual features between some head- and tail classes, the cross-entropy model is prone to misclassify tail samples to similar head classes. Therefore, to alleviate the influence of the confusion on model performance and improve the classification of tail classes, we propose a Similarity Window Reweighting and Margin (SWRM) algorithm, where the SWRM consists of Similarity Window Reweighting (SWR) and Similarity Window Margin (SWM) algorithms. For the confusable head- and tail classes, SWR assigns larger weights to tail classes and smaller weights to head classes. Therefore, the model can enlarge the importance of tail classes and effectively improve their classification. Moreover, SWR considers the difference in label frequency and the impact of category similarity simultaneously, so that the weight coefficients are more reasonable and efficacious. SWM generates adaptive margins that are proportional to the ratio of the classifier’s weight norm, thus promoting the learning of tail classifier with small weight norm. Our SWRM effectively eliminates the confusion between head- and tail classes and alleviates the misclassification issues. Extensive experiments on three long-tailed datasets, i.e., CIFAR100-LT, ImageNet-LT, and Places-LT, verify our proposed method’s effectiveness and superiority over comparative methods.