DeepAI AI Chat
Log In Sign Up

Learning by Cheating

12/27/2019
by   Dian Chen, et al.
26

Vision-based urban driving is hard. The autonomous system needs to learn to perceive the world and act in it. We show that this challenging learning problem can be simplified by decomposing it into two stages. We first train an agent that has access to privileged information. This privileged agent cheats by observing the ground-truth layout of the environment and the positions of all traffic participants. In the second stage, the privileged agent acts as a teacher that trains a purely vision-based sensorimotor agent. The resulting sensorimotor agent does not have access to any privileged information and does not cheat. This two-stage training procedure is counter-intuitive at first, but has a number of important advantages that we analyze and empirically demonstrate. We use the presented approach to train a vision-based autonomous driving system that substantially outperforms the state of the art on the CARLA benchmark and the recent NoCrash benchmark. Our approach achieves, for the first time, 100 sets a new record on the NoCrash benchmark, and reduces the frequency of infractions by an order of magnitude compared to the prior state of the art. For the video that summarizes this work, see https://youtu.be/u9ZCxxD-UUw

READ FULL TEXT
05/03/2021

Learning to drive from a world on rails

We learn an interactive vision-based driving policy from pre-recorded dr...
02/17/2022

CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-based Autonomous Urban Driving

Vision-based autonomous urban driving in dense traffic is quite challeng...
04/08/2021

A Bayesian Approach to Reinforcement Learning of Vision-Based Vehicular Control

In this paper, we present a state-of-the-art reinforcement learning meth...
05/19/2023

Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer

To achieve fully autonomous driving, vehicles must be capable of continu...
09/19/2018

Combined Image- and World-Space Tracking in Traffic Scenes

Tracking in urban street scenes plays a central role in autonomous syste...
09/22/2021

Domain Generalization for Vision-based Driving Trajectory Generation

One of the challenges in vision-based driving trajectory generation is d...

Code Repositories

LearningByCheating

(CoRL 2019) Driving in CARLA using waypoint prediction and two-stage imitation learning


view repo

2020_CARLA_challenge

"Learning by Cheating" (CoRL 2019) submission for the 2020 CARLA Challenge


view repo

carla-birdeye-view

Bird-eye's view for CARLA simulator


view repo

LearningByCheating

Driving in CARLA using waypoints and two-stage imitation learning


view repo