Refining Manually-Designed Symbol Grounding and High-Level Planning by Policy Gradients

by   Takuya Hiraoka, et al.

Hierarchical planners that produce interpretable and appropriate plans are desired, especially in its application to supporting human decision making. In the typical development of the hierarchical planners, higher-level planners and symbol grounding functions are manually created, and this manual creation requires much human effort. In this paper, we propose a framework that can automatically refine symbol grounding functions and a high-level planner to reduce human effort for designing these modules. In our framework, symbol grounding and high-level planning, which are based on manually-designed knowledge bases, are modeled with semi-Markov decision processes. A policy gradient method is then applied to refine the modules, in which two terms for updating the modules are considered. The first term, called a reinforcement term, contributes to updating the modules to improve the overall performance of a hierarchical planner to produce appropriate plans. The second term, called a penalty term, contributes to keeping refined modules consistent with the manually-designed original modules. Namely, it keeps the planner, which uses the refined modules, producing interpretable plans. We perform preliminary experiments to solve the Mountain car problem, and its results show that a manually-designed high-level planner and symbol grounding function were successfully refined by our framework.


page 1

page 2

page 3

page 4


Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning

We address one-shot imitation learning, where the goal is to execute a p...

PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making

Reinforcement learning and symbolic planning have both been used to buil...

Towards Stable Symbol Grounding with Zero-Suppressed State AutoEncoder

While classical planning has been an active branch of AI, its applicabil...

Autonomous Driving at Intersections: A Critical-Turning-Point Approach for Left Turns

Left-turn planning is one of the formidable challenges for autonomous ve...

HATP: An HTN Planner for Robotics

Hierarchical Task Network (HTN) planning is a popular approach that cuts...

Catch Planner: Catching High-Speed Targets in the Flight

Catching high-speed targets in the flight is a complex and typical highl...

Jump Operator Planning: Goal-Conditioned Policy Ensembles and Zero-Shot Transfer

In Hierarchical Control, compositionality, abstraction, and task-transfe...

Please sign up or login with your details

Forgot password? Click here to reset