DeepAI AI Chat
Log In Sign Up

An Empirical Approach For Probing the Definiteness of Kernels

07/10/2018
by   Martin Zaefferer, et al.
TH Köln
0

Models like support vector machines or Gaussian process regression often require positive semi-definite kernels. These kernels may be based on distance functions. While definiteness is proven for common distances and kernels, a proof for a new kernel may require too much time and effort for users who simply aim at practical usage. Furthermore, designing definite distances or kernels may be equally intricate. Finally, models can be enabled to use indefinite kernels. This may deteriorate the accuracy or computational cost of the model. Hence, an efficient method to determine definiteness is required. We propose an empirical approach. We show that sampling as well as optimization with an evolutionary algorithm may be employed to determine definiteness. We provide a proof-of-concept with 16 different distance measures for permutations. Our approach allows to disprove definiteness if a respective counter-example is found. It can also provide an estimate of how likely it is to obtain indefinite kernel matrices. This provides a simple, efficient tool to decide whether additional effort should be spent on designing/selecting a more suitable kernel or algorithm.

READ FULL TEXT
10/09/2015

On the Definiteness of Earth Mover's Distance Yields and Its Relation to Set Intersection

Positive definite kernels are an important tool in machine learning that...
09/06/2012

Solving Support Vector Machines in Reproducing Kernel Banach Spaces with Positive Definite Functions

In this paper we solve support vector machines in reproducing kernel Ban...
06/07/2015

Generalized Spectral Kernels

In this paper we propose a family of tractable kernels that is dense in ...
05/30/2020

Generalizing Random Fourier Features via Generalized Measures

We generalize random Fourier features, that usually require kernel funct...
06/12/2015

Optimal γ and C for ε-Support Vector Regression with RBF Kernels

The objective of this study is to investigate the efficient determinatio...
09/27/2021

Learning from Small Samples: Transformation-Invariant SVMs with Composition and Locality at Multiple Scales

Motivated by the problem of learning when the number of training samples...