Safety Guaranteed Manipulation Based on Reinforcement Learning Planner and Model Predictive Control Actor

by   Zhenshan Bing, et al.

Deep reinforcement learning (RL) has been endowed with high expectations in tackling challenging manipulation tasks in an autonomous and self-directed fashion. Despite the significant strides made in the development of reinforcement learning, the practical deployment of this paradigm is hindered by at least two barriers, namely, the engineering of a reward function and ensuring the safety guaranty of learning-based controllers. In this paper, we address these challenging limitations by proposing a framework that merges a reinforcement learning [columns=fixed]planner that is trained using sparse rewards with a model predictive controller (MPC) [columns=fixed]actor, thereby offering a safe policy. On the one hand, the RL [columns=fixed]planner learns from sparse rewards by selecting intermediate goals that are easy to achieve in the short term and promising to lead to target goals in the long term. On the other hand, the MPC [columns=fixed]actor takes the suggested intermediate goals from the RL [columns=fixed]planner as the input and predicts how the robot's action will enable it to reach that goal while avoiding any obstacles over a short period of time. We evaluated our method on four challenging manipulation tasks with dynamic obstacles and the results demonstrate that, by leveraging the complementary strengths of these two components, the agent can solve manipulation tasks in complex, dynamic environments safely with a 100% success rate. Videos are available at <>.


page 1

page 6

page 8


Complex Robotic Manipulation via Graph-Based Hindsight Goal Generation

Reinforcement learning algorithms such as hindsight experience replay (H...

Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments

Deep reinforcement learning (RL) agents are able to learn contact-rich m...

Actor-Critic Model Predictive Control

Despite its success, Model Predictive Control (MPC) often requires inten...

Deep Reinforcement Learning with Adjustments

Deep reinforcement learning (RL) algorithms can learn complex policies t...

Event-Triggered Model Predictive Control with Deep Reinforcement Learning for Autonomous Driving

Event-triggered model predictive control (eMPC) is a popular optimal con...

Follow the Object: Curriculum Learning for Manipulation Tasks with Imagined Goals

Learning robot manipulation through deep reinforcement learning in envir...

Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians

Robotic navigation in environments shared with other robots or humans re...

Please sign up or login with your details

Forgot password? Click here to reset