DeepAI AI Chat
Log In Sign Up

Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control

by   Yangchen Pan, et al.

Recent work has shown that reinforcement learning (RL) is a promising approach to control dynamical systems described by partial differential equations (PDE). This paper shows how to use RL to tackle more general PDE control problems that have continuous high-dimensional action spaces with spatial relationship among action dimensions. In particular, we propose the concept of action descriptors, which encode regularities among spatially-extended action dimensions and enable the agent to control high-dimensional action PDEs. We provide theoretical evidence suggesting that this approach can be more sample efficient compared to a conventional approach that treats each action dimension separately and does not explicitly exploit the spatial regularity of the action space. The action descriptor approach is then used within the deep deterministic policy gradient algorithm. Experiments on two PDE control problems, with up to 256-dimensional continuous actions, show the advantage of the proposed approach over the conventional one.


Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations

In many areas, such as the physical sciences, life sciences, and finance...

Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning

We present a convolutional framework which significantly reduces the com...

Learning a model is paramount for sample efficiency in reinforcement learning control of PDEs

The goal of this paper is to make a strong point for the usage of dynami...

Online identification and control of PDEs via Reinforcement Learning methods

We focus on the control of unknown Partial Differential Equations (PDEs)...

Discrete Sequential Prediction of Continuous Actions for Deep RL

It has long been assumed that high dimensional continuous control proble...