How to induce regularization in generalized linear models: A guide to reparametrizing gradient flow

08/09/2023
by   Hung-Hsu Chou, et al.
0

In this work, we analyze the relation between reparametrizations of gradient flow and the induced implicit bias on general linear models, which encompass various basic classification and regression tasks. In particular, we aim at understanding the influence of the model parameters - reparametrization, loss, and link function - on the convergence behavior of gradient flow. Our results provide user-friendly conditions under which the implicit bias can be well-described and convergence of the flow is guaranteed. We furthermore show how to use these insights for designing reparametrization functions that lead to specific implicit biases like ℓ_p- or trigonometric regularizers.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset