Hashing embeddings of optimal dimension, with applications to linear least squares

by   Coralia Cartis, et al.

The aim of this paper is two-fold: firstly, to present subspace embedding properties for s-hashing sketching matrices, with s≥ 1, that are optimal in the projection dimension m of the sketch, namely, m=𝒪(d), where d is the dimension of the subspace. A diverse set of results are presented that address the case when the input matrix has sufficiently low coherence (thus removing the log^2 d factor dependence in m, in the low-coherence result of Bourgain et al (2015) at the expense of a smaller coherence requirement); how this coherence changes with the number s of column nonzeros (allowing a scaling of √(s) of the coherence bound), or is reduced through suitable transformations (when considering hashed – instead of subsampled – coherence reducing transformations such as randomised Hadamard). Secondly, we apply these general hashing sketching results to the special case of Linear Least Squares (LLS), and develop Ski-LLS, a generic software package for these problems, that builds upon and improves the Blendenpik solver on dense input and the (sequential) LSRN performance on sparse problems. In addition to the hashing sketching improvements, we add suitable linear algebra tools for rank-deficient and for sparse problems that lead Ski-LLS to outperform not only sketching-based routines on randomly generated input, but also state of the art direct solver SPQR and iterative code HSL on certain subsets of the sparse Florida matrix collection; namely, on least squares problems that are significantly overdetermined, or moderately sparse, or difficult.


page 1

page 2

page 3

page 4


Sparse graph based sketching for fast numerical linear algebra

In recent years, a variety of randomized constructions of sketching matr...

Guarantees for the Kronecker Fast Johnson-Lindenstrauss Transform Using a Coherence and Sampling Argument

In the recent paper [Jin, Kolda Ward, arXiv:1909.04801], it is prove...

Tight Bounds for ℓ_p Oblivious Subspace Embeddings

An ℓ_p oblivious subspace embedding is a distribution over r × n matrice...

Fully Understanding the Hashing Trick

Feature hashing, also known as the hashing trick, introduced by Weinber...

A Weighted Generalized Coherence Approach for Sensing Matrix Design

As compared to using randomly generated sensing matrices, optimizing the...

Optimal Algorithms for Linear Algebra in the Current Matrix Multiplication Time

We study fundamental problems in linear algebra, such as finding a maxim...

Please sign up or login with your details

Forgot password? Click here to reset