The interplay between system identification and machine learning

by   Gianluigi Pillonetto, et al.

Learning from examples is one of the key problems in science and engineering. It deals with function reconstruction from a finite set of direct and noisy samples. Regularization in reproducing kernel Hilbert spaces (RKHSs) is widely used to solve this task and includes powerful estimators such as regularization networks. Recent achievements include the proof of the statistical consistency of these kernel- based approaches. Parallel to this, many different system identification techniques have been developed but the interaction with machine learning does not appear so strong yet. One reason is that the RKHSs usually employed in machine learning do not embed the information available on dynamic systems, e.g. BIBO stability. In addition, in system identification the independent data assumptions routinely adopted in machine learning are never satisfied in practice. This paper provides new results which strengthen the connection between system identification and machine learning. Our starting point is the introduction of RKHSs of dynamic systems. They contain functionals over spaces defined by system inputs and allow to interpret system identification as learning from examples. In both linear and nonlinear settings, it is shown that this perspective permits to derive in a relatively simple way conditions on RKHS stability (i.e. the property of containing only BIBO stable systems or predictors), also facilitating the design of new kernels for system identification. Furthermore, we prove the convergence of the regularized estimator to the optimal predictor under conditions typical of dynamic systems.


page 1

page 2

page 3

page 4


Absolute integrability of Mercer kernels is only sufficient for RKHS stability

Reproducing kernel Hilbert spaces (RKHSs) are special Hilbert spaces in ...

Mathematical foundations of stable RKHSs

Reproducing kernel Hilbert spaces (RKHSs) are key spaces for machine lea...

Kernel absolute summability is only sufficient for RKHS stability

Regularized approaches have been successfully applied to linear system i...

Total stability of kernel methods

Regularized empirical risk minimization using kernels and their correspo...

Identification of stable models via nonparametric prediction error methods

A new Bayesian approach to linear system identification has been propose...

A new kernel-based approach for overparameterized Hammerstein system identification

In this paper we propose a new identification scheme for Hammerstein sys...

Generalized system identification with stable spline kernels

Regularized least-squares approaches have been successfully applied to l...

Please sign up or login with your details

Forgot password? Click here to reset