PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning

by   Aleksandra Faust, et al.

We present PRM-RL, a hierarchical method for long-range navigation task completion that combines sampling-based path planning with reinforcement learning (RL) agents. The RL agents learn short-range, point-to-point navigation policies that capture robot dynamics and task constraints without knowledge of the large-scale topology, while the sampling-based planners provide an approximate map of the space of possible configurations of the robot from which collision-free trajectories feasible for the RL agents can be identified. The same RL agents are used to control the robot under the direction of the planning, enabling long-range navigation. We use the Probabilistic Roadmaps (PRMs) for the sampling-based planner. The RL agents are constructed using feature-based and deep neural net policies in continuous state and action spaces. We evaluate PRM-RL on two navigation tasks with non-trivial robot dynamics: end-to-end differential drive indoor navigation in office environments, and aerial cargo delivery in urban environments with load displacement constraints. These evaluations included both simulated environments and on-robot tests. Our results show improvement in navigation task completion over both RL agents on their own and traditional sampling-based planners. In the indoor navigation task, PRM-RL successfully completes up to 215 meters long trajectories under noisy sensor conditions, and the aerial cargo delivery completes flights over 1000 meters without violating the task constraints in an environment 63 million times larger than used in training.


page 1

page 3

page 8


Long-Range Indoor Navigation with PRM-RL

Long-range indoor navigation requires guiding robots with noisy sensors ...

Hierarchies of Planning and Reinforcement Learning for Robot Navigation

Solving robotic navigation tasks via reinforcement learning (RL) is chal...

Control Transformer: Robot Navigation in Unknown Environments through PRM-Guided Return-Conditioned Sequence Modeling

Learning long-horizon tasks such as navigation has presented difficult c...

Generalizable Wireless Navigation through Physics-Informed Reinforcement Learning in Wireless Digital Twin

The growing focus on indoor robot navigation utilizing wireless signals ...

Decomposing the Prediction Problem; Autonomous Navigation by neoRL Agents

Navigating the world is a fundamental ability for any living entity. Acc...

RoCUS: Robot Controller Understanding via Sampling

As robots are deployed in complex situations, engineers and end users mu...

Robot Representing and Reasoning with Knowledge from Reinforcement Learning

Reinforcement learning (RL) agents aim at learning by interacting with a...

Please sign up or login with your details

Forgot password? Click here to reset