Nonconvex Factorization and Manifold Formulations are Almost Equivalent in Low-rank Matrix Optimization

by   Yuetian Luo, et al.

In this paper, we consider the geometric landscape connection of the widely studied manifold and factorization formulations in low-rank positive semidefinite (PSD) and general matrix optimization. We establish an equivalence on the set of first-order stationary points (FOSPs) and second-order stationary points (SOSPs) between the manifold and the factorization formulations. We further give a sandwich inequality on the spectrum of Riemannian and Euclidean Hessians at FOSPs, which can be used to transfer more geometric properties from one formulation to another. Similarities and differences on the landscape connection under the PSD case and the general case are discussed. To the best of our knowledge, this is the first geometric landscape connection between the manifold and the factorization formulations for handling rank constraints. In the general low-rank matrix optimization, the landscape connection of two factorization formulations (unregularized and regularized ones) is also provided. By applying these geometric landscape connections, we are able to solve unanswered questions in literature and establish stronger results in the applications on geometric analysis of phase retrieval, well-conditioned low-rank matrix optimization, and the role of regularization in factorization arising from machine learning and signal processing.


page 1

page 2

page 3

page 4


On Geometric Connections of Embedded and Quotient Geometries in Riemannian Fixed-rank Matrix Optimization

In this paper, we propose a general procedure for establishing the lands...

Global Optimality in Distributed Low-rank Matrix Factorization

We study the convergence of a variant of distributed gradient descent (D...

Landscape Correspondence of Empirical and Population Risks in the Eigendecomposition Problem

Spectral methods include a family of algorithms related to the eigenvect...

Statistical Query Complexity of Manifold Estimation

This paper studies the statistical query (SQ) complexity of estimating d...

Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points

Motivated by the emerging role of interpolating machines in signal proce...

Finding stationary points on bounded-rank matrices: A geometric hurdle and a smooth remedy

We consider the problem of provably finding a stationary point of a smoo...

Please sign up or login with your details

Forgot password? Click here to reset