SACSoN: Scalable Autonomous Data Collection for Social Navigation

by   Noriaki Hirose, et al.

Machine learning provides a powerful tool for building socially compliant robotic systems that go beyond simple predictive models of human behavior. By observing and understanding human interactions from past experiences, learning can enable effective social navigation behaviors directly from data. However, collecting navigation data in human-occupied environments may require teleoperation or continuous monitoring, making the process prohibitively expensive to scale. In this paper, we present a scalable data collection system for vision-based navigation, SACSoN, that can autonomously navigate around pedestrians in challenging real-world environments while encouraging rich interactions. SACSoN uses visual observations to observe and react to humans in its vicinity. It couples this visual understanding with continual learning and an autonomous collision recovery system that limits the involvement of a human operator, allowing for better dataset scaling. We use a this system to collect the SACSoN dataset, the largest-of-its-kind visual navigation dataset of autonomous robots operating in human-occupied spaces, spanning over 75 hours and 4000 rich interactions with humans. Our experiments show that collecting data with a novel objective that encourages interactions, leads to significant improvements in downstream tasks such as inferring pedestrian dynamics and learning socially compliant navigation behaviors. We make videos of our autonomous data collection system and the SACSoN dataset publicly available on our project page.


page 1

page 3

page 4

page 6

page 9


Toward Human-Like Social Robot Navigation: A Large-Scale, Multi-Modal, Social Human Navigation Dataset

Humans are well-adept at navigating public spaces shared with others, wh...

Towards Rich, Portable, and Large-Scale Pedestrian Data Collection

Recently, pedestrian behavior research has shifted towards machine learn...

Elly: A Real-Time Failure Recovery and Data Collection System for Robotic Manipulation

Even the most robust autonomous behaviors can fail. The goal of this res...

Offline Reinforcement Learning for Visual Navigation

Reinforcement learning can enable robots to navigate to distant goals wh...

Following Social Groups: Socially Compliant Autonomous Navigation in Dense Crowds

In densely populated environments, socially compliant navigation is crit...

One-Shot Informed Robotic Visual Search in the Wild

We consider the task of underwater robot navigation for the purpose of c...

Utilizing wearable technology to characterize and facilitate occupant collaborations in flexible workspaces

Hybrid working strategies have become, and will continue to be, the norm...

Please sign up or login with your details

Forgot password? Click here to reset