Knowledge-Guided Exploration in Deep Reinforcement Learning
share
This paper proposes a new method to drastically speed up deep reinforcement learning (deep RL) training for problems that have the property of state-action permissibility (SAP). Two types of permissibility are defined under SAP. The first type says that after an action a_t is performed in a state s_t and the agent has reached the new state s_t+1, the agent can decide whether a_t is permissible or not permissible in s_t. The second type says that even without performing a_t in s_t, the agent can already decide whether a_t is permissible or not in s_t. An action is not permissible in a state if the action can never lead to an optimal solution and thus should not be tried (over and over again). We incorporate the proposed SAP property and encode action permissibility knowledge into two state-of-the-art deep RL algorithms to guide their state-action exploration together with a virtual stopping strategy. Results show that the SAP-based guidance can markedly speed up RL training.
READ FULL TEXT