Generalized system identification with stable spline kernels

by   Aleksandr Y. Aravkin, et al.

Regularized least-squares approaches have been successfully applied to linear system identification. Recent approaches use quadratic penalty terms on the unknown impulse response defined by stable spline kernels, which control model space complexity by leveraging regularity and bounded-input bounded-output stability. This paper extends linear system identification to a wide class of nonsmooth stable spline estimators, where regularization functionals and data misfits can be selected from a rich set of piecewise linear quadratic penalties. This class encompasses the 1-norm, huber, and vapnik, in addition to the least-squares penalty, and the approach allows linear inequality constraints on the unknown impulse response. We develop a customized interior point solver for the entire class of proposed formulations. By representing penalties through their conjugates, we allow a simple interface that enables the user to specify any piecewise linear quadratic penalty for misfit and regularizer, together with inequality constraints on the response. The solver is locally quadratically convergent, with O(n2(m+n)) arithmetic operations per iteration, for n impulse response coefficients and m output measurements. In the system identification context, where n << m, IPsolve is competitive with available alternatives, illustrated by a comparison with TFOCS and libSVM. The modeling framework is illustrated with a range of numerical experiments, featuring robust formulations for contaminated data, relaxation systems, and nonnegativity and unimodality constraints on the impulse response. Incorporating constraints yields significant improvements in system identification. The solver used to obtain the results is distributed via an open source code repository.


page 1

page 2

page 3

page 4


Linear system identification using stable spline kernels and PLQ penalties

The classical approach to linear system identification is given by param...

On the Maximum Entropy Property of the First-Order Stable Spline Kernel and its Implications

A new nonparametric approach for system identification has been recently...

Score Engineered Robust Least Squares Regression

In other FICO Technical Papers, I have shown how to fit Generalized Addi...

On the estimation of initial conditions in kernel-based system identification

Recent developments in system identification have brought attention to r...

Mathematical foundations of stable RKHSs

Reproducing kernel Hilbert spaces (RKHSs) are key spaces for machine lea...

Bayesian and regularization approaches to multivariable linear system identification: the role of rank penalties

Recent developments in linear system identification have proposed the us...

The interplay between system identification and machine learning

Learning from examples is one of the key problems in science and enginee...

Please sign up or login with your details

Forgot password? Click here to reset