Deliberative Acting, Online Planning and Learning with Hierarchical Operational Models

by   Sunandita Patra, et al.

The most common representation formalisms for automated planning are descriptive models that abstractly describe what the actions do and are tailored for effciently computing the next state(s) in a state-transition system. However, real-world acting requires operational models that describe how to do things, with rich control structures for closed-loop online decision-making in a dynamic environment. To use a different action model for planning than the one used for acting causes problems with combining acting and planning, in particular for the development and consistency verification of the different models. As an alternative, we define and implement an integrated acting-and-planning system in which both planning and acting use the same operational models, which are written in a general-purpose hierarchical task-oriented language offering rich control structures. The acting component, called Reactive Acting Engine (RAE), is inspired by the well-known PRS system, except that instead of being purely reactive, it can get advice from the planner. Our planner uses a UCT-like Monte Carlo Tree Search procedure, called UPOM (UCT Procedure for Operational Models), whose rollouts are simulations of the actor's operational models. We also present learning strategies for use with RAE and UPOM that acquire, from online acting experiences and/or simulated planning results, a mapping from decision contexts to method instances as well as a heuristic function to guide UPOM. Our experimental results show that UPOM and our learning strategies significantly improve the acting efficiency and robustness of RAE. We discuss the asymptotic convergence of UPOM by mapping its search space to an MDP.


Integrating Acting, Planning and Learning in Hierarchical Operational Models

We present new planning and learning algorithms for RAE, the Refinement ...

Towards Blended Reactive Planning and Acting using Behavior Trees

In this paper, we study the problem of using a planning algorithm to aut...

FAPE: a Constraint-based Planner for Generative and Hierarchical Temporal Planning

Temporal planning offers numerous advantages when based on an expressive...

SMarTplan: a Task Planner for Smart Factories

Smart factories are on the verge of becoming the new industrial paradigm...

Focused Model-Learning and Planning for Non-Gaussian Continuous State-Action Systems

We introduce a framework for model learning and planning in stochastic d...

Habits of Mind: Reusing Action Sequences for Efficient Planning

When we exercise sequences of actions, their execution becomes more flue...

Agile Satellite Planning for Multi-Payload Observations for Earth Science

We present planning challenges, methods and preliminary results for a ne...

Please sign up or login with your details

Forgot password? Click here to reset