Near-Optimal Entrywise Sampling of Numerically Sparse Matrices
share
Many real-world data sets are sparse or almost sparse. One method to measure this for a matrix A∈ℝ^n× n is the numerical sparsity, denoted 𝗇𝗌(A), defined as the minimum k≥ 1 such that a_1/a_2 ≤√(k) for every row and every column a of A. This measure of a is smooth and is clearly only smaller than the number of non-zeros in the row/column a. The seminal work of Achlioptas and McSherry [2007] has put forward the question of approximating an input matrix A by entrywise sampling. More precisely, the goal is to quickly compute a sparse matrix à satisfying A - Ã_2 ≤ϵA_2 (i.e., additive spectral approximation) given an error parameter ϵ>0. The known schemes sample and rescale a small fraction of entries from A. We propose a scheme that sparsifies an almost-sparse matrix A – it produces a matrix à with O(ϵ^-2𝗇𝗌(A) · nln n) non-zero entries with high probability. We also prove that this upper bound on 𝗇𝗇𝗓(Ã) is tight up to logarithmic factors. Moreover, our upper bound improves when the spectrum of A decays quickly (roughly replacing n with the stable rank of A). Our scheme can be implemented in time O(𝗇𝗇𝗓(A)) when A_2 is given. Previously, a similar upper bound was obtained by Achlioptas et. al [2013] but only for a restricted class of inputs that does not even include symmetric or covariance matrices. Finally, we demonstrate two applications of these sampling techniques, to faster approximate matrix multiplication, and to ridge regression by using sparse preconditioners.
READ FULL TEXT