The Foreseeable Future: Self-Supervised Learning to Predict Dynamic Scenes for Indoor Navigation

by   Hugues Thomas, et al.

We present a method for generating, predicting, and using Spatiotemporal Occupancy Grid Maps (SOGM), which embed future semantic information of real dynamic scenes. We present an auto-labeling process that creates SOGMs from noisy real navigation data. We use a 3D-2D feedforward architecture, trained to predict the future time steps of SOGMs, given 3D lidar frames as input. Our pipeline is entirely self-supervised, thus enabling lifelong learning for real robots. The network is composed of a 3D back-end that extracts rich features and enables the semantic segmentation of the lidar frames, and a 2D front-end that predicts the future information embedded in the SOGM representation, potentially capturing the complexities and uncertainties of real-world multi-agent, multi-future interactions. We also design a navigation system that uses these predicted SOGMs within planning, after they have been transformed into Spatiotemporal Risk Maps (SRMs). We verify our navigation system's abilities in simulation, validate it on a real robot, study SOGM predictions on real data in various circumstances, and provide a novel indoor 3D lidar dataset, collected during our experiments, which includes our automated annotations.


page 1

page 6

page 7

page 8

page 10

page 12

page 14

page 15


Learning Spatiotemporal Occupancy Grid Maps for Lifelong Navigation in Dynamic Scenes

We present a novel method for generating, predicting, and using Spatiote...

Self-Supervised Learning of Lidar Segmentation for Autonomous Indoor Navigation

We present a self-supervised learning approach for the semantic segmenta...

Object Goal Navigation with End-to-End Self-Supervision

A household robot should be able to navigate to target locations without...

A Self-Supervised Miniature One-Shot Texture Segmentation (MOSTS) Model for Real-Time Robot Navigation and Embedded Applications

Determining the drivable area, or free space segmentation, is critical f...

2D LiDAR Map Prediction via Estimating Motion Flow with GRU

It is a significant problem to predict the 2D LiDAR map at next moment f...

Learning to Navigate from Simulation via Spatial and Semantic Information Synthesis

While training an end-to-end navigation network in the real world is usu...

Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions

We focus on the problem of predicting future states of entities in compl...

Please sign up or login with your details

Forgot password? Click here to reset