Optimal Feature Transport for Cross-View Image Geo-Localization

by   Yujiao Shi, et al.

This paper addresses the problem of cross-view image based localization, where the geographic location of a ground-level street-view query image is estimated by matching it against a large scale aerial map (e.g., a high-resolution satellite image). State-of-the-art deep-learning based methods tackle this problem as deep metric learning which aims to learn global feature representations of the scene seen by the two different views. Despite promising results are obtained by such deep metric learning methods, they, however, fail to exploit a crucial cue relevant for localization, namely, the spatial layout of local features. Moreover, little attention is paid to the obvious domain gap (between aerial view and ground view) in the context of cross-view localization. This paper proposes a novel Cross-View Feature Transport (CVFT) technique to explicitly establish cross-view domain transfer that facilitates feature alignment between ground and aerial images. Specifically, we implement the CVFT as a network layer, which transports features from one domain to the other, leading to more meaningful feature similarity comparison. Our model is differentiable and can be learned end-to-end. Experiments on large-scale datasets have demonstrated that our method has remarkably boosted the state-of-the-art cross-view localization performance, e.g., on the CVUSA dataset, with significant improvements for top-1 recall from 40.79 and for top-10 from 76.36 art [14]. We expect the key insight of the paper (i.e., explicitly handling domain difference via domain transport) will prove to be useful for other similar problems in computer vision as well.


page 1

page 5

page 6

page 8

page 11

page 12


Lending Orientation to Neural Networks for Cross-view Geo-localization

This paper studies image-based geo-localization (IBL) problem using grou...

Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching

Cross-view geo-localization is the problem of estimating the position an...

Image-based Geolocalization by Ground-to-2.5D Map Matching

We study the image-based geolocalization problem that aims to locate gro...

Deep Phase Correlation for End-to-End Heterogeneous Sensor Measurements Matching

The crucial step for localization is to match the current observation to...

Mutual Generative Transformer Learning for Cross-view Geo-localization

Cross-view geo-localization (CVGL), which aims to estimate the geographi...

Bridging the Domain Gap for Ground-to-Aerial Image Matching

The visual entities in cross-view images exhibit drastic domain changes ...

Cross-view Geo-localization with Evolving Transformer

In this work, we address the problem of cross-view geo-localization, whi...

Please sign up or login with your details

Forgot password? Click here to reset