Approximate Guarantees for Dictionary Learning

by   Aditya Bhaskara, et al.

In the dictionary learning (or sparse coding) problem, we are given a collection of signals (vectors in R^d), and the goal is to find a "basis" in which the signals have a sparse (approximate) representation. The problem has received a lot of attention in signal processing, learning, and theoretical computer science. The problem is formalized as factorizing a matrix X (d × n) (whose columns are the signals) as X = AY, where A has a prescribed number m of columns (typically m ≪ n), and Y has columns that are k-sparse (typically k ≪ d). Most of the known theoretical results involve assuming that the columns of the unknown A have certain incoherence properties, and that the coefficient matrix Y has random (or partly random) structure. The goal of our work is to understand what can be said in the absence of such assumptions. Can we still find A and Y such that X ≈ AY? We show that this is possible, if we allow violating the bounds on m and k by appropriate factors that depend on k and the desired approximation. Our results rely on an algorithm for what we call the threshold correlation problem, which turns out to be related to hypercontractive norms of matrices. We also show that our algorithmic ideas apply to a setting in which some of the columns of X are outliers, thus giving similar guarantees even in this challenging setting.


page 1

page 2

page 3

page 4


New Algorithms for Learning Incoherent and Overcomplete Dictionaries

In sparse recovery we are given a matrix A (the dictionary) and a vector...

NOODL: Provable Online Dictionary Learning and Sparse Coding

We consider the dictionary learning problem, where the aim is to model t...

Towards Learning Sparsely Used Dictionaries with Arbitrary Supports

Dictionary learning is a popular approach for inferring a hidden basis o...

Sparse and spurious: dictionary learning with noise and outliers

A popular approach within the signal processing and machine learning com...

More Algorithms for Provable Dictionary Learning

In dictionary learning, also known as sparse coding, the algorithm is gi...

Local identifiability of l_1-minimization dictionary learning: a sufficient and almost necessary condition

We study the theoretical properties of learning a dictionary from N sign...

A Dictionary Learning Approach for Factorial Gaussian Models

In this paper, we develop a parameter estimation method for factorially ...

Please sign up or login with your details

Forgot password? Click here to reset