How Many Samples is a Good Initial Point Worth?

by   Gavin Zhang, et al.

Given a sufficiently large amount of labeled data, the non-convex low-rank matrix recovery problem contains no spurious local minima, so a local optimization algorithm is guaranteed to converge to a global minimum starting from any initial guess. However, the actual amount of data needed by this theoretical guarantee is very pessimistic, as it must prevent spurious local minima from existing anywhere, including at adversarial locations. In contrast, prior work based on good initial guesses have more realistic data requirements, because they allow spurious local minima to exist outside of a neighborhood of the solution. In this paper, we quantify the relationship between the quality of the initial guess and the corresponding reduction in data requirements. Using the restricted isometry constant as a surrogate for sample complexity, we compute a sharp threshold number of samples needed to prevent each specific point on the optimization landscape from becoming a spurious local minima. Optimizing the threshold over regions of the landscape, we see that, for initial points not too close to the ground truth, a linear improvement in the quality of the initial guess amounts to a constant factor improvement in the sample complexity.


page 1

page 2

page 3

page 4


Sharp Restricted Isometry Bounds for the Inexistence of Spurious Local Minima in Nonconvex Matrix Recovery

Nonconvex matrix recovery is known to contain no spurious local minima u...

Sharp Restricted Isometry Property Bounds for Low-rank Matrix Recovery Problems with Corrupted Measurements

In this paper, we study a general low-rank matrix recovery problem with ...

Sharp Global Guarantees for Nonconvex Low-Rank Matrix Recovery in the Overparameterized Regime

We prove that it is possible for nonconvex low-rank matrix recovery to c...

Learning One-hidden-layer Neural Networks with Landscape Design

We consider the problem of learning a one-hidden-layer neural network: w...

Noisy Low-rank Matrix Optimization: Geometry of Local Minima and Convergence Rate

This paper is concerned with low-rank matrix optimization, which has fou...

Existence of local minima of a minimal 2D pose-graph SLAM problem

In this paper, we show that for a minimal pose-graph problem, even in th...

On the Distribution of Minima in Intrinsic-Metric Rotation Averaging

Rotation Averaging is a non-convex optimization problem that determines ...

Please sign up or login with your details

Forgot password? Click here to reset