The asymptotic spectrum of the Hessian of DNN throughout training

10/01/2019
by   Arthur Jacot, et al.
0

The dynamics of DNNs during gradient descent is described by the so-called Neural Tangent Kernel (NTK). In this article, we show that the NTK allows one to gain precise insight into the Hessian of the cost of DNNs: we obtain a full characterization of the asymptotics of the spectrum of the Hessian, at initialization and during training.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset