On Kernel Derivative Approximation with Random Fourier Features

by   Zoltan Szabo, et al.
Penn State University
Ecole Polytechnique

Random Fourier features (RFF) represent one of the most popular and wide-spread techniques in machine learning to scale up kernel algorithms. Despite the numerous successful applications of RFFs, unfortunately, quite little is understood theoretically on their optimality and limitations of their performance. To the best of our knowledge, the only existing areas where precise statistical-computational trade-offs have been established are approximation of kernel values, kernel ridge regression, and kernel principal component analysis. Our goal is to spark the investigation of optimality of RFF-based approximations in tasks involving not only function values but derivatives, which naturally lead to optimization problems with kernel derivatives. Particularly, in this paper, we focus on the approximation quality of RFFs for kernel derivatives and prove that the existing finite-sample guarantees can be improved exponentially in terms of the domain where they hold, using recent tools from unbounded empirical process theory. Our result implies that the same approximation guarantee is achievable for kernel derivatives using RFF as for kernel values.


page 1

page 2

page 3

page 4


Optimal Rates for Random Fourier Features

Kernel methods represent one of the most powerful tools in machine learn...

Statistical Optimality and Computational Efficiency of Nyström Kernel PCA

Kernel methods provide an elegant framework for developing nonlinear lea...

Streaming Kernel PCA with Õ(√(n)) Random Features

We study the statistical and computational aspects of kernel principal c...

On the Error of Random Fourier Features

Kernel methods give powerful, flexible, and theoretically grounded appro...

Kernel Methods and their derivatives: Concept and perspectives for the Earth system sciences

Kernel methods are powerful machine learning techniques which implement ...

Statistical and Computational Trade-Offs in Kernel K-Means

We investigate the efficiency of k-means in terms of both statistical an...

A Linear Approximation to the chi^2 Kernel with Geometric Convergence

We propose a new analytical approximation to the χ^2 kernel that converg...

Please sign up or login with your details

Forgot password? Click here to reset