In-context learning for model-free system identification

by   Marco Forgione, et al.

In traditional system identification, we estimate a model of an unknown dynamical system based on given input/output sequences and available physical knowledge. Yet, is it also possible to understand the intricacies of dynamical systems not solely from their input/output patterns, but by observing the behavior of other systems within the same class? This central question drives the study presented in this paper. In response to this query, we introduce a novel paradigm for system identification, addressing two primary tasks: one-step-ahead prediction and multi-step simulation. Unlike conventional methods, we do not directly estimate a model for the specific system. Instead, we pretrain a meta model that represents a class of dynamical systems. This meta model is trained from a potentially infinite stream of synthetic data, generated by systems randomly extracted from a certain distribution. At its core, the meta model serves as an implicit representation of the main characteristics of a class of dynamical systems. When provided with a brief context from a new system - specifically, a short input/output sequence - the meta model implicitly discerns its dynamics, enabling predictions of its behavior. The proposed approach harnesses the power of Transformer architectures, renowned for their in-context learning capabilities in Natural Language Processing tasks. For one-step prediction, a GPT-like decoder-only architecture is utilized, whereas the simulation problem employs an encoder-decoder structure. Initial experimental results affirmatively answer our foundational question, opening doors to fresh research avenues in system identification.


page 1

page 2

page 3

page 4


Dynamical System Parameter Identification using Deep Recurrent Cell Networks

In this paper, we investigate the parameter identification problem in dy...

Deep Representation Learning for Dynamical Systems Modeling

Proper states' representations are the key to the successful dynamics mo...

Model structures and fitting criteria for system identification with neural networks

This paper focuses on the identification of dynamical systems with tailo...

Can Transformers Learn Optimal Filtering for Unknown Systems?

Transformers have demonstrated remarkable success in natural language pr...

Tensorized Transformer for Dynamical Systems Modeling

The identification of nonlinear dynamics from observations is essential ...

Artificial Neural Network Based Prediction of Optimal Pseudo-Damping and Meta-Damping in Oscillatory Fractional Order Dynamical Systems

This paper investigates typical behaviors like damped oscillations in fr...

Customizing Sequence Generation with Multi-Task Dynamical Systems

Dynamical system models (including RNNs) often lack the ability to adapt...

Please sign up or login with your details

Forgot password? Click here to reset