Zero-Shot Uncertainty-Aware Deployment of Simulation Trained Policies on Real-World Robots

by   Krishan Rana, et al.

While deep reinforcement learning (RL) agents have demonstrated incredible potential in attaining dexterous behaviours for robotics, they tend to make errors when deployed in the real world due to mismatches between the training and execution environments. In contrast, the classical robotics community have developed a range of controllers that can safely operate across most states in the real world given their explicit derivation. These controllers however lack the dexterity required for complex tasks given limitations in analytical modelling and approximations. In this paper, we propose Bayesian Controller Fusion (BCF), a novel uncertainty-aware deployment strategy that combines the strengths of deep RL policies and traditional handcrafted controllers. In this framework, we can perform zero-shot sim-to-real transfer, where our uncertainty based formulation allows the robot to reliably act within out-of-distribution states by leveraging the handcrafted controller while gaining the dexterity of the learned system otherwise. We show promising results on two real-world continuous control tasks, where BCF outperforms both the standalone policy and controller, surpassing what either can achieve independently. A supplementary video demonstrating our system is provided at


page 6

page 7


Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Robotics

We present Bayesian Controller Fusion (BCF): a hybrid control strategy t...

Retro-RL: Reinforcing Nominal Controller With Deep Reinforcement Learning for Tilting-Rotor Drones

Studies that broaden drone applications into complex tasks require a sta...

Quad2Plane: An Intermediate Training Procedure for Online Exploration in Aerial Robotics via Receding Horizon Control

Data driven robotics relies upon accurate real-world representations to ...

Robotic Table Tennis: A Case Study into a High Speed Learning System

We present a deep-dive into a real-world robotic learning system that, i...

Uncertainty Aware Learning from Demonstrations in Multiple Contexts using Bayesian Neural Networks

Diversity of environments is a key challenge that causes learned robotic...

How to Train your Quadrotor: A Framework for Consistently Smooth and Responsive Flight Control via Reinforcement Learning

We focus on the problem of reliably training Reinforcement Learning (RL)...

Zero-Shot Terrain Generalization for Visual Locomotion Policies

Legged robots have unparalleled mobility on unstructured terrains. Howev...

Please sign up or login with your details

Forgot password? Click here to reset