Fast Algorithms for Monotone Lower Subsets of Kronecker Least Squares Problems

by   Osman Asif Malik, et al.
University of Colorado Boulder
wells fargo
Berkeley Lab

Approximate solutions to large least squares problems can be computed efficiently using leverage score-based row-sketches, but directly computing the leverage scores, or sampling according to them with naive methods, still requires an expensive manipulation and processing of the design matrix. In this paper we develop efficient leverage score-based sampling methods for matrices with certain Kronecker product-type structure; in particular we consider matrices that are monotone lower column subsets of Kronecker product matrices. Our discussion is general, encompassing least squares problems on infinite domains, in which case matrices formally have infinitely many rows. We briefly survey leverage score-based sampling guarantees from the numerical linear algebra and approximation theory communities, and follow this with efficient algorithms for sampling when the design matrix has Kronecker-type structure. Our numerical examples confirm that sketches based on exact leverage score sampling for our class of structured matrices achieve superior residual compared to approximate leverage score sampling methods.


Sketching Matrix Least Squares via Leverage Scores Estimates

We consider the matrix least squares problem of the form 𝐀𝐗-𝐁_F^2 where ...

Fast Exact Leverage Score Sampling from Khatri-Rao Products with Applications to Tensor Decomposition

We present a data structure to randomly sample rows from the Khatri-Rao ...

Max-plus statistical leverage scores

The statistical leverage scores of a complex matrix A∈C^n× d record the ...

Sharper Bounds for ℓ_p Sensitivity Sampling

In large scale machine learning, random sampling is a popular way to app...

On Fast Leverage Score Sampling and Optimal Learning

Leverage score sampling provides an appealing way to perform approximate...

Gradient Coding through Iterative Block Leverage Score Sampling

We generalize the leverage score sampling sketch for ℓ_2-subspace embedd...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset