FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing

by   Kyle Stachowicz, et al.

We present a system that enables an autonomous small-scale RC car to drive aggressively from visual observations using reinforcement learning (RL). Our system, FastRLAP (faster lap), trains autonomously in the real world, without human interventions, and without requiring any simulation or expert demonstrations. Our system integrates a number of important components to make this possible: we initialize the representations for the RL policy and value function from a large prior dataset of other robots navigating in other environments (at low speed), which provides a navigation-relevant representation. From here, a sample-efficient online RL method uses a single low-speed user-provided demonstration to determine the desired driving course, extracts a set of navigational checkpoints, and autonomously practices driving through these checkpoints, resetting automatically on collision or failure. Perhaps surprisingly, we find that with appropriate initialization and choice of algorithm, our system can learn to drive over a variety of racing courses with less than 20 minutes of online training. The resulting policies exhibit emergent aggressive driving skills, such as timing braking and acceleration around turns and avoiding areas which impede the robot's motion, approaching the performance of a human driver using a similar first-person interface over the course of training.


page 1

page 2

page 3

page 6

page 7

page 8

page 9


Integrating Imitation Learning with Human Driving Data into Reinforcement Learning to Improve Training Efficiency for Autonomous Driving

Two current methods used to train autonomous cars are reinforcement lear...

Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration

Safe Reinforcement Learning (RL) aims to find a policy that achieves hig...

Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving

Currently, urban autonomous driving remains challenging because of the c...

Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skills

Efficient and effective exploration in continuous space is a central pro...

Formula RL: Deep Reinforcement Learning for Autonomous Racing using Telemetry Data

This paper explores the use of reinforcement learning (RL) models for au...

Learning by Watching

When in a new situation or geographical location, human drivers have an ...

DRIVE: Data-driven Robot Input Vector Exploration

An accurate motion model is a fundamental component of most autonomous n...

Please sign up or login with your details

Forgot password? Click here to reset