Bridging Active Exploration and Uncertainty-Aware Deployment Using Probabilistic Ensemble Neural Network Dynamics

by   Taekyung Kim, et al.

In recent years, learning-based control in robotics has gained significant attention due to its capability to address complex tasks in real-world environments. With the advances in machine learning algorithms and computational capabilities, this approach is becoming increasingly important for solving challenging control problems in robotics by learning unknown or partially known robot dynamics. Active exploration, in which a robot directs itself to states that yield the highest information gain, is essential for efficient data collection and minimizing human supervision. Similarly, uncertainty-aware deployment has been a growing concern in robotic control, as uncertain actions informed by the learned model can lead to unstable motions or failure. However, active exploration and uncertainty-aware deployment have been studied independently, and there is limited literature that seamlessly integrates them. This paper presents a unified model-based reinforcement learning framework that bridges these two tasks in the robotics control domain. Our framework uses a probabilistic ensemble neural network for dynamics learning, allowing the quantification of epistemic uncertainty via Jensen-Renyi Divergence. The two opposing tasks of exploration and deployment are optimized through state-of-the-art sampling-based MPC, resulting in efficient collection of training data and successful avoidance of uncertain state-action spaces. We conduct experiments on both autonomous vehicles and wheeled robots, showing promising results for both exploration and deployment.


page 1

page 9

page 10

page 11


Active Exploration for Robotic Manipulation

Robotic manipulation stands as a largely unsolved problem despite signif...

Safe Navigation in Unstructured Environments by Minimizing Uncertainty in Control and Perception

Uncertainty in control and perception poses challenges for autonomous ve...

Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Control

Uncertainty quantification is one of the central challenges for machine ...

Particle MPC for Uncertain and Learning-Based Control

As robotic systems move from highly structured environments to open worl...

Optimistic Active Exploration of Dynamical Systems

Reinforcement learning algorithms commonly seek to optimize policies for...

PlaNet of the Bayesians: Reconsidering and Improving Deep Planning Network by Incorporating Bayesian Inference

In the present paper, we propose an extension of the Deep Planning Netwo...

Physics-informed reinforcement learning via probabilistic co-adjustment functions

Reinforcement learning of real-world tasks is very data inefficient, and...

Please sign up or login with your details

Forgot password? Click here to reset