Seeing the Big Picture: Deep Embedding with Contextual Evidences

by   Liang Zheng, et al.

In the Bag-of-Words (BoW) model based image retrieval task, the precision of visual matching plays a critical role in improving retrieval performance. Conventionally, local cues of a keypoint are employed. However, such strategy does not consider the contextual evidences of a keypoint, a problem which would lead to the prevalence of false matches. To address this problem, this paper defines "true match" as a pair of keypoints which are similar on three levels, i.e., local, regional, and global. Then, a principled probabilistic framework is established, which is capable of implicitly integrating discriminative cues from all these feature levels. Specifically, the Convolutional Neural Network (CNN) is employed to extract features from regional and global patches, leading to the so-called "Deep Embedding" framework. CNN has been shown to produce excellent performance on a dozen computer vision tasks such as image classification and detection, but few works have been done on BoW based image retrieval. In this paper, firstly we show that proper pre-processing techniques are necessary for effective usage of CNN feature. Then, in the attempt to fit it into our model, a novel indexing structure called "Deep Indexing" is introduced, which dramatically reduces memory usage. Extensive experiments on three benchmark datasets demonstrate that, the proposed Deep Embedding method greatly promotes the retrieval accuracy when CNN feature is integrated. We show that our method is efficient in terms of both memory and time cost, and compares favorably with the state-of-the-art methods.


page 1

page 2

page 3

page 4

page 5

page 8

page 10


From Selective Deep Convolutional Features to Compact Binary Representations for Image Retrieval

Convolutional Neural Network (CNN) is a very powerful approach to extrac...

A Comparison of CNN and Classic Features for Image Retrieval

Feature detectors and descriptors have been successfully used for variou...

A Dense-Depth Representation for VLAD descriptors in Content-Based Image Retrieval

The recent advances brought by deep learning allowed to improve the perf...

Selective Deep Convolutional Features for Image Retrieval

Convolutional Neural Network (CNN) is a very powerful approach to extrac...

Indexing of CNN Features for Large Scale Image Search

Convolutional neural network (CNN) features which represent images with ...

A Feature Embedding Strategy for High-level CNN representations from Multiple ConvNets

Following the rapidly growing digital image usage, automatic image categ...

Who's Afraid of Adversarial Queries? The Impact of Image Modifications on Content-based Image Retrieval

An adversarial query is an image that has been modified to disrupt conte...

Please sign up or login with your details

Forgot password? Click here to reset