Journal article
Sketch-based image retrieval with deep visual semantic descriptor
Pattern recognition, Vol.76, pp.537-548
04/2018
DOI: 10.1016/j.patcog.2017.11.032
Abstract
•Novel Real-time SBIR is built by employing multimodal deep sketch-image matching.•Deep Visual Semantic Descriptor is created to bridge sketch-image multimodal gap.•Sketch-like Transformation is established to improve sketch-image resemblance.•Re-ranking Optimization is introduced to characterize sketch-image correlation.
Sketch-based Image Retrieval (SBIR) has received a lot of attentions recently. In this paper we aim to enhance SBIR with deep visual semantic descriptor and related optimization mechanisms. Our scheme significantly differs from other earlier work in: 1) A feature representation via deep visual semantic descriptor is established to bridge the gap between sketches and images, which can encode both low-level local features and high-level semantic features; 2) A clustering-based re-ranking optimization is introduced to further improve SBIR by dynamically adjusting the correlations of images in the ranking list. The main contribution of our work is that we effectively apply the deep visual semantic descriptor to enable deep sketch-image matching, which has provided a more reasonable base for us to fuse local low-level visual features with high-level semantic features by determining an optimal correlated mapping. Our experiments on a large number of public data have obtained very positive results.
Details
- Title: Subtitle
- Sketch-based image retrieval with deep visual semantic descriptor
- Creators
- Fei Huang - School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, ChinaCheng Jin - School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, ChinaYuejie Zhang - School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, ChinaKangnian Weng - School of Information Management and Engineering, Shanghai University of Finance and Economics, Shanghai, ChinaTao Zhang - School of Information Management and Engineering, Shanghai University of Finance and Economics, Shanghai, ChinaWeiguo Fan - School of Accounting and Information Systems, Virginia Tech, Blacksburg, VA, USA
- Resource Type
- Journal article
- Publication Details
- Pattern recognition, Vol.76, pp.537-548
- Publisher
- Elsevier Ltd
- DOI
- 10.1016/j.patcog.2017.11.032
- ISSN
- 0031-3203
- eISSN
- 1873-5142
- Grant note
- DOI: 10.13039/501100001809, name: National Natural Science Foundation of China, award: 61572140, 61672165
- Language
- English
- Date published
- 04/2018
- Academic Unit
- Business Analytics
- Record Identifier
- 9984083219402771
Metrics
19 Record Views