Centralized Cooperative Exploration Policy for Continuous Control Tasks

by   Chao Li, et al.

The deep reinforcement learning (DRL) algorithm works brilliantly on solving various complex control tasks. This phenomenal success can be partly attributed to DRL encouraging intelligent agents to sufficiently explore the environment and collect diverse experiences during the agent training process. Therefore, exploration plays a significant role in accessing an optimal policy for DRL. Despite recent works making great progress in continuous control tasks, exploration in these tasks has remained insufficiently investigated. To explicitly encourage exploration in continuous control tasks, we propose CCEP (Centralized Cooperative Exploration Policy), which utilizes underestimation and overestimation of value functions to maintain the capacity of exploration. CCEP first keeps two value functions initialized with different parameters, and generates diverse policies with multiple exploration styles from a pair of value functions. In addition, a centralized policy framework ensures that CCEP achieves message delivery between multiple policies, furthermore contributing to exploring the environment cooperatively. Extensive experimental results demonstrate that CCEP achieves higher exploration capacity. Empirical analysis shows diverse exploration styles in the learned policies by CCEP, reaping benefits in more exploration regions. And this exploration capacity of CCEP ensures it outperforms the current state-of-the-art methods across multiple continuous control tasks shown in experiments.


page 6

page 7

page 14


Exploration-efficient Deep Reinforcement Learning with Demonstration Guidance for Robot Control

Although deep reinforcement learning (DRL) algorithms have made importan...

Normality-Guided Distributional Reinforcement Learning for Continuous Control

Learning a predictive model of the mean return, or value function, plays...

Particle-Based Adaptive Discretization for Continuous Control using Deep Reinforcement Learning

Learning controls in high-dimensional continuous action spaces, such as ...

Diverse Exploration for Fast and Safe Policy Improvement

We study an important yet under-addressed problem of quickly and safely ...

Minimum Coverage Sets for Training Robust Ad Hoc Teamwork Agents

Robustly cooperating with unseen agents and human partners presents sign...

Please sign up or login with your details

Forgot password? Click here to reset