GuaRD: Guaranteed robustness of image retrieval system under data distortion turbulence

Abstract

Image search systems could be endangered by adversarial attacks and data perturbations. The image retrieval system can be compromised either by distorting the query or hacking the ranking system. However, existing literature primarily discusses attack methods, whereas the research on countermeasures to defend against such adversarial attacks is rare. As a defense mechanism against the intrusions, quality assessment can complement existing image retrieval systems. “GuaRD” is proposed as an end-to-end framework that uses the quality metric as a weighted-regularization term. Proper utilization and balance of the two features could lead to reliable and robust ranking; the original image is assigned a higher rank while the distorted image is assigned a relatively lower rank. Meanwhile, the primary goal of the image retrieval system is to prioritize searching the relevant images. Therefore, the use of leveraged features should not compromise the accuracy of the system. To evaluate the generality of the framework, we conducted three experiments on two image quality assessment(IQA) benchmarks (Waterloo and PieAPP). For the first two tests, GuaRD achieved enhanced performance than the existing model: the mean reciprocal rank(mRR) value of the original image predictions increased by 61%, and the predictions for the distorted input query decreased by 18%. The third experiment was conducted to analyze the mean average precision (mAP) score of the system to verify the accuracy of the retrieval system. The results indicated little deviation in performance between the tested methods, and the score was not effected or slightly decreased by 0.9% after the GuaRD was applied. Therefore, GuaRD is a novel and robust framework that can act as a defense mechanism for data distortions.