Neural Collapse with Cross-Entropy Loss

12/15/2020

∙

by   Jianfeng Lu, et al.

∙

0

∙

share

We consider the variational problem of cross-entropy loss with n feature vectors on a unit hypersphere in ℝ^d. We prove that when d ≥ n - 1, the global minimum is given by the simplex equiangular tight frame, which justifies the neural collapse behavior. We also show a connection with the frame potential of Benedetto Fickus.