Convergence Analysis of the Rank-Restricted Soft SVD Algorithm

by   Mahendra Panagoda, et al.

The soft SVD is a robust matrix decomposition algorithm and a key component of matrix completion methods. However, computing the soft SVD for large sparse matrices is often impractical using conventional numerical methods for the SVD due to large memory requirements. The Rank-Restricted Soft SVD (RRSS) algorithm introduced by Hastie et al. addressed this issue by sequentially computing low-rank SVDs that easily fit in memory. We analyze the convergence of the standard RRSS algorithm and we give examples where the standard algorithm does not converge. We show that convergence requires a modification of the standard algorithm, and is related to non-uniqueness of the SVD. Our modification specifies a consistent choice of sign for the left singular vectors of the low-rank SVDs in the iteration. Under these conditions, we prove linear convergence of the singular vectors using a technique motivated by alternating subspace iteration. We then derive a fixed point iteration for the evolution of the singular values and show linear convergence to the soft thresholded singular values of the original matrix. This last step requires a perturbation result for fixed point iterations which may be of independent interest.


page 1

page 2

page 3

page 4


Ranky : An Approach to Solve Distributed SVD on Large Sparse Matrices

Singular Value Decomposition (SVD) is a well studied research topic in m...

A Fast Implementation of Singular Value Thresholding Algorithm using Recycling Rank Revealing Randomized Singular Value Decomposition

In this paper, we present a fast implementation of the Singular Value Th...

LazySVD: Even Faster SVD Decomposition Yet Without Agonizing Pain

We study k-SVD that is to obtain the first k singular vectors of a matri...

Fast Stochastic Algorithms for SVD and PCA: Convergence Properties and Convexity

We study the convergence properties of the VR-PCA algorithm introduced b...

Depth from Small Motion using Rank-1 Initialization

Depth from Small Motion (DfSM) (Ha et al., 2016) is particularly interes...

Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls

We propose a rank-k variant of the classical Frank-Wolfe algorithm to so...

An answer to an open question in the incremental SVD

Incremental singular value decomposition (SVD) was proposed by Brand to ...

Please sign up or login with your details

Forgot password? Click here to reset