DeepAI AI Chat
Log In Sign Up

Learning Data-adaptive Nonparametric Kernels

by   Fanghui Liu, et al.
Nanjing University
Tsinghua University

Traditional kernels or their combinations are often not sufficiently flexible to fit the data in complicated practical tasks. In this paper, we present a Data-Adaptive Nonparametric Kernel (DANK) learning framework by imposing an adaptive matrix on the kernel/Gram matrix in an entry-wise strategy. Since we do not specify the formulation of the adaptive matrix, each entry in it can be directly and flexibly learned from the data. Therefore, the solution space of the learned kernel is largely expanded, which makes DANK flexible to adapt to the data. Specifically, the proposed kernel learning framework can be seamlessly embedded to support vector machines (SVM) and support vector regression (SVR), which has the capability of enlarging the margin between classes and reducing the model generalization error. Theoretically, we demonstrate that the objective function of our devised model is gradient-Lipschitz continuous. Thereby, the training process for kernel and parameter learning in SVM/SVR can be efficiently optimized in a unified framework. Further, to address the scalability issue in DANK, a decomposition-based scalable approach is developed, of which the effectiveness is demonstrated by both empirical studies and theoretical guarantees. Experimentally, our method outperforms other representative kernel learning based algorithms on various classification and regression benchmark datasets.


Snacks: a fast large-scale kernel SVM solver

Kernel methods provide a powerful framework for non parametric learning....

Complex Support Vector Machines for Regression and Quaternary Classification

The paper presents a new framework for complex Support Vector Regression...

AML-SVM: Adaptive Multilevel Learning with Support Vector Machines

The support vector machines (SVM) is one of the most widely used and pra...

Adaptive Kernel Value Caching for SVM Training

Support Vector Machines (SVMs) can solve structured multi-output learnin...

TrIK-SVM : an alternative decomposition for kernel methods in Krein spaces

The proposed work aims at proposing a alternative kernel decomposition i...

Generalization Properties of hyper-RKHS and its Application to Out-of-Sample Extensions

Hyper-kernels endowed by hyper-Reproducing Kernel Hilbert Space (hyper-R...

Learning from Distributions via Support Measure Machines

This paper presents a kernel-based discriminative learning framework on ...