A Novel M-Estimator for Robust PCA

by   Teng Zhang, et al.

We study the basic problem of robust subspace recovery. That is, we assume a data set that some of its points are sampled around a fixed subspace and the rest of them are spread in the whole ambient space, and we aim to recover the fixed underlying subspace. We first estimate "robust inverse sample covariance" by solving a convex minimization procedure; we then recover the subspace by the bottom eigenvectors of this matrix (their number correspond to the number of eigenvalues close to 0). We guarantee exact subspace recovery under some conditions on the underlying data. Furthermore, we propose a fast iterative algorithm, which linearly converges to the matrix minimizing the convex problem. We also quantify the effect of noise and regularization and discuss many other practical and theoretical issues for improving the subspace recovery in various settings. When replacing the sum of terms in the convex energy function (that we minimize) with the sum of squares of terms, we obtain that the new minimizer is a scaled version of the inverse sample covariance (when exists). We thus interpret our minimizer and its subspace (spanned by its bottom eigenvectors) as robust versions of the empirical inverse covariance and the PCA subspace respectively. We compare our method with many other algorithms for robust PCA on synthetic and real data sets and demonstrate state-of-the-art speed and accuracy.


page 31

page 33


Distributed Robust Subspace Recovery

We study Robust Subspace Recovery (RSR) in distributed settings. We cons...

Fast, Robust and Non-convex Subspace Recovery

This work presents a fast and non-convex algorithm for robust subspace r...

A Well-Tempered Landscape for Non-convex Robust Subspace Recovery

We present a mathematical analysis of a non-convex energy landscape for ...

On the Subspace of Image Gradient Orientations

We introduce the notion of Principal Component Analysis (PCA) of image g...

Stochastic and Private Nonconvex Outlier-Robust PCA

We develop theoretically guaranteed stochastic methods for outlier-robus...

lp-Recovery of the Most Significant Subspace among Multiple Subspaces with Outliers

We assume data sampled from a mixture of d-dimensional linear subspaces ...

Least sum of squares of trimmed residuals regression

In the famous least sum of trimmed squares (LTS) of residuals estimator ...

Please sign up or login with your details

Forgot password? Click here to reset