Implicit Regularization in Matrix Factorization

05/25/2017
by   Suriya Gunasekar, et al.
0

We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix X with gradient descent on a factorization of X. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a full dimensional factorization converges to the minimum nuclear norm solution.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset