On Column Selection in Approximate Kernel Canonical Correlation Analysis

by   Weiran Wang, et al.
Toyota Technological Institute at Chicago

We study the problem of column selection in large-scale kernel canonical correlation analysis (KCCA) using the Nyström approximation, where one approximates two positive semi-definite kernel matrices using "landmark" points from the training set. When building low-rank kernel approximations in KCCA, previous work mostly samples the landmarks uniformly at random from the training set. We propose novel strategies for sampling the landmarks non-uniformly based on a version of statistical leverage scores recently developed for kernel ridge regression. We study the approximation accuracy of the proposed non-uniform sampling strategy, develop an incremental algorithm that explores the path of approximation ranks and facilitates efficient model selection, and derive the kernel stability of out-of-sample mapping for our method. Experimental results on both synthetic and real-world datasets demonstrate the promise of our method.


page 1

page 2

page 3

page 4


Kernel Ridge Regression Using Importance Sampling with Application to Seismic Response Prediction

Scalable kernel methods, including kernel ridge regression, often rely o...

Recursive Sampling for the Nyström Method

We give the first algorithm for kernel Nyström approximation that runs i...

Diversity sampling is an implicit regularization for kernel methods

Kernel methods have achieved very good performance on large scale regres...

Relating Leverage Scores and Density using Regularized Christoffel Functions

Statistical leverage scores emerged as a fundamental tool for matrix ske...

On Fast Leverage Score Sampling and Optimal Learning

Leverage score sampling provides an appealing way to perform approximate...

A General Scoring Rule for Randomized Kernel Approximation with Application to Canonical Correlation Analysis

Random features has been widely used for kernel approximation in large-s...

Nyström landmark sampling and regularized Christoffel functions

Selecting diverse and important items from a large set is a problem of i...

Please sign up or login with your details

Forgot password? Click here to reset