Learning to Reach, Swim, Walk and Fly in One Trial: Data-Driven Control with Scarce Data and Side Information

by   Franck Djeumou, et al.

We develop a learning-based control algorithm for unknown dynamical systems under very severe data limitations. Specifically, the algorithm has access to streaming data only from a single and ongoing trial. Despite the scarcity of data, we show – through a series of examples – that the algorithm can provide performance comparable to reinforcement learning algorithms trained over millions of environment interactions. It accomplishes such performance by effectively leveraging various forms of side information on the dynamics to reduce the sample complexity. Such side information typically comes from elementary laws of physics and qualitative properties of the system. More precisely, the algorithm approximately solves an optimal control problem encoding the system's desired behavior. To this end, it constructs and refines a differential inclusion that contains the unknown vector field of the dynamics. The differential inclusion, used in an interval Taylor-based method, enables to over-approximate the set of states the system may reach. Theoretically, we establish a bound on the suboptimality of the approximate solution with respect to the case of known dynamics. We show that the longer the trial or the more side information is available, the tighter the bound. Empirically, experiments in a high-fidelity F-16 aircraft simulator and MuJoCo's environments such as the Reacher, Swimmer, and Cheetah illustrate the algorithm's effectiveness.


Deep Learning of Koopman Representation for Control

We develop a data-driven, model-free approach for the optimal control of...

Optimistic Active Exploration of Dynamical Systems

Reinforcement learning algorithms commonly seek to optimize policies for...

Multifidelity Reinforcement Learning with Control Variates

In many computational science and engineering applications, the output o...

Learning-Based Optimal Control with Performance Guarantees for Unknown Systems with Latent States

As control engineering methods are applied to increasingly complex syste...

SpReME: Sparse Regression for Multi-Environment Dynamic Systems

Learning dynamical systems is a promising avenue for scientific discover...

Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

Optimal control (OC) algorithms such as Differential Dynamic Programming...

Learning Hybrid Dynamics Models With Simulator-Informed Latent States

Dynamics model learning deals with the task of inferring unknown dynamic...

Please sign up or login with your details

Forgot password? Click here to reset