More Algorithms for Provable Dictionary Learning

by   Sanjeev Arora, et al.

In dictionary learning, also known as sparse coding, the algorithm is given samples of the form y = Ax where x∈R^m is an unknown random sparse vector and A is an unknown dictionary matrix in R^n× m (usually m > n, which is the overcomplete case). The goal is to learn A and x. This problem has been studied in neuroscience, machine learning, visions, and image processing. In practice it is solved by heuristic algorithms and provable algorithms seemed hard to find. Recently, provable algorithms were found that work if the unknown feature vector x is √(n)-sparse or even sparser. Spielman et al. DBLP:journals/jmlr/SpielmanWW12 did this for dictionaries where m=n; Arora et al. AGM gave an algorithm for overcomplete (m >n) and incoherent matrices A; and Agarwal et al. DBLP:journals/corr/AgarwalAN13 handled a similar case but with weaker guarantees. This raised the problem of designing provable algorithms that allow sparsity ≫√(n) in the hidden vector x. The current paper designs algorithms that allow sparsity up to n/poly( n). It works for a class of matrices where features are individually recoverable, a new notion identified in this paper that may motivate further work. The algorithm runs in quasipolynomial time because they use limited enumeration.


page 1

page 2

page 3

page 4


New Algorithms for Learning Incoherent and Overcomplete Dictionaries

In sparse recovery we are given a matrix A (the dictionary) and a vector...

NOODL: Provable Online Dictionary Learning and Sparse Coding

We consider the dictionary learning problem, where the aim is to model t...

Provably Accurate Double-Sparse Coding

Sparse coding is a crucial subroutine in algorithms for various signal p...

Simple, Efficient, and Neural Algorithms for Sparse Coding

Sparse coding is a basic task in many fields including signal processing...

Subgradient Descent Learns Orthogonal Dictionaries

This paper concerns dictionary learning, i.e., sparse coding, a fundamen...

Approximate Guarantees for Dictionary Learning

In the dictionary learning (or sparse coding) problem, we are given a co...

Faster provable sieving algorithms for the Shortest Vector Problem and the Closest Vector Problem on lattices in ℓ_p norm

In this paper we give provable sieving algorithms for the Shortest Vecto...

Please sign up or login with your details

Forgot password? Click here to reset