Distributed Learning with Random Features

by   Jian Li, et al.

Distributed learning and random projections are the most common techniques in large scale nonparametric statistical learning. In this paper, we study the generalization properties of kernel ridge regression using both distributed methods and random features. Theoretical analysis shows the combination remarkably reduces computational cost while preserving the optimal generalization accuracy under standard assumptions. In a benign case, O(√(N)) partitions and O(√(N)) random features are sufficient to achieve O(1/N) learning rate, where N is the labeled sample size. Further, we derive more refined results by using additional unlabeled data to enlarge the number of partitions and by generating features in a data-dependent way to reduce the number of random features.


page 5

page 6

page 7


Generalization Properties of Learning with Random Features

We study the generalization properties of ridge regression with random f...

Distributed Learning with Dependent Samples

This paper focuses on learning rate analysis of distributed kernel ridge...

FALKON: An Optimal Large Scale Kernel Method

Kernel methods provide a principled way to perform non linear, nonparame...

Generalization Guarantees for Sparse Kernel Approximation with Entropic Optimal Features

Despite their success, kernel methods suffer from a massive computationa...

Interpolation and Learning with Scale Dependent Kernels

We study the learning properties of nonparametric ridge-less least squar...

Sharp Analysis of Random Fourier Features in Classification

We study the theoretical properties of random Fourier features classific...

ParK: Sound and Efficient Kernel Ridge Regression by Feature Space Partitions

We introduce ParK, a new large-scale solver for kernel ridge regression....

Please sign up or login with your details

Forgot password? Click here to reset