Algorithms for Learning Kernels Based on Centered Alignment

by   Corinna Cortes, et al.

This paper presents new and effective algorithms for learning kernels. In particular, as shown by our empirical results, these algorithms consistently outperform the so-called uniform combination solution that has proven to be difficult to improve upon in the past, as well as other algorithms for learning kernels based on convex combinations of base kernels in both classification and regression. Our algorithms are based on the notion of centered alignment which is used as a similarity measure between kernels or kernel matrices. We present a number of novel algorithmic, theoretical, and empirical results for learning kernels based on our notion of centered alignment. In particular, we describe efficient algorithms for learning a maximum alignment kernel by showing that the problem can be reduced to a simple QP and discuss a one-stage algorithm for learning both a kernel and a hypothesis based on that kernel using an alignment-based regularization. Our theoretical results include a novel concentration bound for centered alignment between kernel matrices, the proof of the existence of effective predictors for kernels with high alignment, both for classification and for regression, and the proof of stability-based generalization bounds for a broad family of algorithms for learning kernels based on centered alignment. We also report the results of experiments with our centered alignment-based algorithms in both classification and regression.


page 1

page 2

page 3

page 4


Multi-Task Learning Using Neighborhood Kernels

This paper introduces a new and effective algorithm for learning kernels...

Ensembles of Kernel Predictors

This paper examines the problem of learning with a finite and possibly l...

Gaussian RBF Centered Kernel Alignment (CKA) in the Large Bandwidth Limit

We prove that Centered Kernel Alignment (CKA) based on a Gaussian RBF ke...

L2 Regularization for Learning Kernels

The choice of the kernel is critical to the success of many learning alg...

New Generalization Bounds for Learning Kernels

This paper presents several novel generalization bounds for the problem ...

A Framework for an Assessment of the Kernel-target Alignment in Tree Ensemble Kernel Learning

Kernels ensuing from tree ensembles such as random forest (RF) or gradie...

A kernel-independent sum-of-Gaussians method by de la Vallée-Poussin sums

Approximation of interacting kernels by sum of Gaussians (SOG) is freque...

Please sign up or login with your details

Forgot password? Click here to reset