A scaled gradient projection method for Bayesian learning in dynamical systems

by   Silvia Bonettini, et al.

A crucial task in system identification problems is the selection of the most appropriate model class, and is classically addressed resorting to cross-validation or using asymptotic arguments. As recently suggested in the literature, this can be addressed in a Bayesian framework, where model complexity is regulated by few hyperparameters, which can be estimated via marginal likelihood maximization. It is thus of primary importance to design effective optimization methods to solve the corresponding optimization problem. If the unknown impulse response is modeled as a Gaussian process with a suitable kernel, the maximization of the marginal likelihood leads to a challenging nonconvex optimization problem, which requires a stable and effective solution strategy. In this paper we address this problem by means of a scaled gradient projection algorithm, in which the scaling matrix and the steplength parameter play a crucial role to provide a meaning solution in a computational time comparable with second order methods. In particular, we propose both a generalization of the split gradient approach to design the scaling matrix in the presence of box constraints, and an effective implementation of the gradient and objective function. The extensive numerical experiments carried out on several test problems show that our method is very effective in providing in few tenths of a second solutions of the problems with accuracy comparable with state-of-the-art approaches. Moreover, the flexibility of the proposed strategy makes it easily adaptable to a wider range of problems arising in different areas of machine learning, signal processing and system identification.


page 1

page 2

page 3

page 4


Maximum Entropy Vector Kernels for MIMO system identification

Recent contributions have framed linear system identification as a nonpa...

On-line Bayesian System Identification

We consider an on-line system identification setting, in which new data ...

A new kernel-based approach to system identification with quantized output data

In this paper we introduce a novel method for linear system identificati...

Algorithms for solving optimization problems arising from deep neural net models: nonsmooth problems

Machine Learning models incorporating multiple layered learning networks...

Scalarizing Functions in Bayesian Multiobjective Optimization

Scalarizing functions have been widely used to convert a multiobjective ...

Analysis-of-marginal-Tail-Means - a new method for robust parameter optimization

This paper presents a novel method, called Analysis-of-marginal-Tail-Mea...

MaxMin-L2-SVC-NCH: A New Method to Train Support Vector Classifier with the Selection of Model's Parameters

The selection of model's parameters plays an important role in the appli...

Please sign up or login with your details

Forgot password? Click here to reset