Generalizing Random Fourier Features via Generalized Measures

by   Fanghui Liu, et al.
Shanghai Jiao Tong University
KU Leuven

We generalize random Fourier features, that usually require kernel functions to be both stationary and positive definite (PD), to a more general range of non-stationary or/and non-PD kernels, e.g., dot-product kernels on the unit sphere and a linear combination of positive definite kernels. Specifically, we find that the popular neural tangent kernel in two-layer ReLU network, a typical dot-product kernel, is shift-invariant but not positive definite if we consider ℓ_2-normalized data. By introducing the signed measure, we propose a general framework that covers the above kernels by associating them with specific finite Borel measures, i.e., probability distributions. In this manner, we are able to provide the first random features algorithm to obtain unbiased estimation of these kernels. Experiments on several benchmark datasets verify the effectiveness of our algorithm over the existing methods. Last but not least, our work provides a sufficient and necessary condition, which is also computationally implementable, to solve a long-lasting open question: does any indefinite kernel have a positive decomposition?


page 1

page 2

page 3

page 4


Towards Unbiased Random Features with Lower Variance For Stationary Indefinite Kernels

Random Fourier Features (RFF) demonstrate wellappreciated performance in...

Random Fourier Features for Asymmetric Kernels

The random Fourier features (RFFs) method is a powerful and popular tech...

Revisiting Memory Efficient Kernel Approximation: An Indefinite Learning Perspective

Matrix approximations are a key element in large-scale algebraic machine...

Strictly positive definite non-isotropic kernels on two-point homogeneous manifolds: The asymptotic approach

We present sufficient condition for a family of positive definite kernel...

On Translation Invariant Kernels and Screw Functions

We explore the connection between Hilbertian metrics and positive defini...

An Empirical Approach For Probing the Definiteness of Kernels

Models like support vector machines or Gaussian process regression often...

Convolutional Spectral Kernel Learning

Recently, non-stationary spectral kernels have drawn much attention, owi...

Please sign up or login with your details

Forgot password? Click here to reset