Stochastic Approximation Algorithms for Principal Component Analysis

by   Jian Vora, et al.

Principal Component Analysis is a novel way of of dimensionality reduction. This problem essentially boils down to finding the top k eigen vectors of the data covariance matrix. A considerable amount of literature is found on algorithms meant to do so such as an online method be Warmuth and Kuzmin, Matrix Stochastic Gradient by Arora, Oja's method and many others. In this paper we see some of these stochastic approaches to the PCA optimization problem and comment on their convergence and runtime to obtain an epsilon sub-optimal solution. We revisit convex relaxation based methods for stochastic optimization of principal component analysis. While methods that directly solve the non convex problem have been shown to be optimal in terms of statistical and computational efficiency, the methods based on convex relaxation have been shown to enjoy comparable, or even superior, empirical performance. This motivates the need for a deeper formal understanding of the latter.


page 1

page 2

page 3

page 4


Prescriptive PCA: Dimensionality Reduction for Two-stage Stochastic Optimization

In this paper, we consider the alignment between an upstream dimensional...

Covariance Matrix Adaptation Evolution Strategy Assisted by Principal Component Analysis

Over the past decades, more and more methods gain a giant development du...

Exact Guarantees on the Absence of Spurious Local Minima for Non-negative Robust Principal Component Analysis

This work is concerned with the non-negative robust principal component ...

Fair Principal Component Analysis and Filter Design

We consider Fair Principal Component Analysis (FPCA) and search for a lo...

Efficient Algorithms for High-Dimensional Convex Subspace Optimization via Strict Complementarity

We consider optimization problems in which the goal is find a k-dimensio...

Lazy stochastic principal component analysis

Stochastic principal component analysis (SPCA) has become a popular dime...

Online Principal Component Analysis in High Dimension: Which Algorithm to Choose?

In the current context of data explosion, online techniques that do not ...

Please sign up or login with your details

Forgot password? Click here to reset