Complex Scenario Image Retrieval via Deep Similarity-aware Hashing

Abstract

When performing hashing-based image retrieval, it is difficult to learn discriminative hash codes especially for the multi-label, zero-shot and fine-grained settings. This is due to the fact that the similarities vary, even within the same category, under the conditions of complex scenario settings. To address this problem, this study develops a deep similarity-aware hashing method for complex scenario image retrieval (DEPISH). DEPISH more focuses on the samples that are difficult to distinguish from other images (i.e., “difficult samples”), such as images that contain multiple semantics. It dynamically divides attention among samples according to their difficulty levels with a margin weighting strategy. Furthermore, by adding special terms in the model, DEPISH is capable of avoiding the inconsistency between the hash code representation and true similarity among negative samples. In addition, unlike the existing methods that use a pre-defined similarity matrix with fixed values, the DEPISH adopts an adaptive similarity matrix, which accurately captures the various similarities among all samples. The results of our experiment on multiple benchmark datasets containing complex scenarios (i.e., multi-label, zero-shot, and fine-grained datasets) verify the effectiveness of this method.