Global semantics correlation transmitting and learning for sketch-based cross-domain visual retrieval

Author:

Jiao ShichaoORCID,Han Xie,Kuang LiqunORCID,Xiong Fengguang,He Ligang

Abstract

AbstractSketch-based cross-domain visual data retrieval is the process of searching for images or 3D models using sketches as input. Achieving feature alignment is a significantly challenging task due to the high heterogeneity of cross-domain data. However, the alignment process faces significant challenges, such as domain gap, semantic gap, and knowledge gap. The existing methods adopt different ideas for sketch-based image and 3D shape retrieval tasks, one is domain alignment, and the other is semantic alignment. Technically, both tasks verify the accuracy of extracted features. Hence, we propose a method based on the global feature correlation and the feature similarity for multiple sketch-based cross-domain retrieval tasks. Specifically, the data from various modalities are fed into separate feature extractors to generate original features. Then, these features are projected to the shared subspace. Finally, domain consistency learning, semantic consistency learning, feature correlation learning and feature similarity learning are performed jointly to make the projected features modality-invariance. We evaluate our method on multiple benchmark datasets. Where the MAP in Sketchy, TU-Berlin, SHREC 2013 and SHREC 2014 are 0.466, 0.473, 0.860 and 0.816. The extensive experimental results demonstrate the superiority and generalization of the proposed method, compared to the state-of-the-art approaches. The in-depth analyses of various design choices are also provided to gain insight into the effectiveness of the proposed method. The outcomes of this research contribute to advancing the field of sketch-based cross-domain visual data retrieval and are expected to be applied to a variety of applications that require efficient retrieval of cross-domain domain data.

Funder

National Natural Science Foundation of China

National Natural Science Foundation of Shanxi

Transformation Program of Scientific and Technological Achievements of Jiangsu Province

Research Project Supported by Shanxi Scholarship Council of China

Patent Navigator Project of Taiyuan

Foundation of Shanxi Key Laboratory of Machine Vision and Virtual Reality

Publisher

Springer Science and Business Media LLC

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3