Separation Results between Fixed-Kernel and Feature-Learning Probability Metrics

06/10/2021
by   Carles Domingo Enrich, et al.
0

Several works in implicit and explicit generative modeling empirically observed that feature-learning discriminators outperform fixed-kernel discriminators in terms of the sample quality of the models. We provide separation results between probability metrics with fixed-kernel and feature-learning discriminators using the function classes ℱ_2 and ℱ_1 respectively, which were developed to study overparametrized two-layer neural networks. In particular, we construct pairs of distributions over hyper-spheres that can not be discriminated by fixed kernel (ℱ_2) integral probability metric (IPM) and Stein discrepancy (SD) in high dimensions, but that can be discriminated by their feature learning (ℱ_1) counterparts. To further study the separation we provide links between the ℱ_1 and ℱ_2 IPMs with sliced Wasserstein distances. Our work suggests that fixed-kernel discriminators perform worse than their feature learning counterparts because their corresponding metrics are weaker.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset