ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints

by   Dhruv Shah, et al.

Robotic navigation has been approached as a problem of 3D reconstruction and planning, as well as an end-to-end learning problem. However, long-range navigation requires both planning and reasoning about local traversability, as well as being able to utilize information about global geography, in the form of a roadmap, GPS, or other side information, which provides important navigational hints but may be low-fidelity or unreliable. In this work, we propose a learning-based approach that integrates learning and planning, and can utilize side information such as schematic roadmaps, satellite maps and GPS coordinates as a planning heuristic, without relying on them being accurate. Our method, ViKiNG, incorporates a local traversability model, which looks at the robot's current camera observation and a potential subgoal to infer how easily that subgoal can be reached, as well as a heuristic model, which looks at overhead maps and attempts to estimate the distance to the destination for various subgoals. These models are used by a heuristic planner to decide the best next subgoal in order to reach the final destination. Our method performs no explicit geometric reconstruction, utilizing only a topological representation of the environment. Despite having never seen trajectories longer than 80 meters in its training dataset, ViKiNG can leverage its image-based learned controller and goal-directed heuristic to navigate to goals up to 3 kilometers away in previously unseen environments, and exhibit complex behaviors such as probing potential paths and doubling back when they are found to be non-viable. ViKiNG is also robust to unreliable maps and GPS, since the low-level controller ultimately makes decisions based on egocentric image observations, using maps only as planning heuristics. For videos of our experiments, please check out


page 1

page 2

page 3

page 7

page 8

page 9

page 13


Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation

How can a delivery robot navigate reliably to a destination in a new off...

ViNG: Learning Open-World Navigation with Visual Goals

We propose a learning-based navigation system for reaching visually indi...

Lifelong Topological Visual Navigation

The ability for a robot to navigate with only the use of vision is appea...

Learning Local Heuristics for Search-Based Navigation Planning

Graph search planning algorithms for navigation typically rely heavily o...

RECON: Rapid Exploration for Open-World Navigation with Latent Goal Models

We describe a robotic learning system for autonomous navigation in diver...

Planning Beyond the Sensing Horizon Using a Learned Context

Last-mile delivery systems commonly propose the use of autonomous roboti...

Hybrid Imitative Planning with Geometric and Predictive Costs in Off-road Environments

Geometric methods for solving open-world off-road navigation tasks, by l...

Please sign up or login with your details

Forgot password? Click here to reset