Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization

05/20/2023
by   Ting Wu, et al.
0

Models trained with empirical risk minimization (ERM) are revealed to easily rely on spurious correlations, resulting in poor generalization. Group distributionally robust optimization (group DRO) can alleviate this problem by minimizing the worst-case loss over pre-defined groups. While promising, in practice factors like expensive annotations and privacy preclude the availability of group labels. More crucially, when taking a closer look at the failure modes of out-of-distribution generalization, the typical procedure of reweighting in group DRO loses efficiency. Hinged on the limitations, in this work, we reformulate the group DRO framework by proposing Q-Diversity. Characterized by an interactive training mode, Q-Diversity relaxes the group identification from annotation into direct parameterization. Furthermore, a novel mixing strategy across groups is presented to diversify the under-represented groups. In a series of experiments on both synthetic and real-world text classification tasks, results demonstrate that Q-Diversity can consistently improve worst-case accuracy under different distributional shifts, outperforming state-of-the-art alternatives.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/02/2022

AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization

Models trained via empirical risk minimization (ERM) are known to rely o...
10/21/2022

Just Mix Once: Worst-group Generalization by Group Interpolation

Advances in deep learning theory have revealed how average generalizatio...
06/14/2021

Examining and Combating Spurious Features under Distribution Shift

A central goal of machine learning is to learn robust representations th...
08/26/2022

Take One Gram of Neural Features, Get Enhanced Group Robustness

Predictive performance of machine learning models trained with empirical...
06/20/2018

Fairness Without Demographics in Repeated Loss Minimization

Machine learning models (e.g., speech recognizers) are usually trained t...
10/06/2021

Focus on the Common Good: Group Distributional Robustness Follows

We consider the problem of training a classification model with group an...

Please sign up or login with your details

Forgot password? Click here to reset