SC-wLS: Towards Interpretable Feed-forward Camera Re-localization

by   Xin Wu, et al.

Visual re-localization aims to recover camera poses in a known environment, which is vital for applications like robotics or augmented reality. Feed-forward absolute camera pose regression methods directly output poses by a network, but suffer from low accuracy. Meanwhile, scene coordinate based methods are accurate, but need iterative RANSAC post-processing, which brings challenges to efficient end-to-end training and inference. In order to have the best of both worlds, we propose a feed-forward method termed SC-wLS that exploits all scene coordinate estimates for weighted least squares pose regression. This differentiable formulation exploits a weight network imposed on 2D-3D correspondences, and requires pose supervision only. Qualitative results demonstrate the interpretability of learned weights. Evaluations on 7Scenes and Cambridge datasets show significantly promoted performance when compared with former feed-forward counterparts. Moreover, our SC-wLS method enables a new capability: self-supervised test-time adaptation on the weight network. Codes and models are publicly available.


page 2

page 13

page 18


Random Forests versus Neural Networks - What's Best for Camera Localization?

This work addresses the task of camera localization in a known 3D scene ...

Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC

We describe a learning-based system that estimates the camera position a...

Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation

Fully test-time adaptation aims to adapt the network model based on sequ...

Transformer Feed-Forward Layers Are Key-Value Memories

Feed-forward layers constitute two-thirds of a transformer model's param...

Dense Object Reconstruction from RGBD Images with Embedded Deep Shape Representations

Most problems involving simultaneous localization and mapping can nowada...

Safe Crossover of Neural Networks Through Neuron Alignment

One of the main and largely unexplored challenges in evolvingthe weights...

Adaptive Test-Time Augmentation for Low-Power CPU

Convolutional Neural Networks (ConvNets) are trained offline using the f...

Please sign up or login with your details

Forgot password? Click here to reset