Out-of-Distribution Generalization with Maximal Invariant Predictor

by   Masanori Koyama, et al.

Out-of-Distribution (OOD) generalization problem is a problem of seeking the predictor function whose performance in the worst environments is optimal. This paper makes two contributions to OOD problem. We first use the basic results of probability to prove maximal Invariant Predictor(MIP) condition, a theoretical result that can be used to identify the OOD optimal solution. We then use our MIP to derive inner-environmental Gradient Alignment(IGA) algorithm that can be used to help seek the OOD optimal predictor. Previous studies that have investigated the theoretical aspect of the OOD-problem use strong structural assumptions such as causal DAG. However, in cases involving image datasets, for example, the identification of hidden structural relations is itself a difficult problem. Our theoretical results are different from those of many previous studies in that it can be applied to cases in which the underlying structure of a dataset is difficult to analyze. We present an extensive comparison of previous theoretical approaches to the OODproblems based on the assumptions they make. We also present an extension of the colored-MNIST that can more accurately represent the pathological OOD situation than the original version, and demonstrate the superiority of IGA over previous methods on both the original and the extended version of Colored-MNIST.


Robust Reconfigurable Intelligent Surfaces via Invariant Risk and Causal Representations

In this paper, the problem of robust reconfigurable intelligent surface ...

Causally Invariant Predictor with Shift-Robustness

This paper proposes an invariant causal predictor that is robust to dist...

Learning The MMSE Channel Predictor

We present a neural network based predictor which is derived by starting...

Learning Optimal Features via Partial Invariance

Learning models that are robust to test-time distribution shifts is a ke...

Learning-Augmented k-means Clustering

k-means clustering is a well-studied problem due to its wide applicabili...

On a simplified version of Hadamard's maximal determinant problem

Hadamard's maximal determinant problem consists in finding the maximal v...

Distributionally Robust Data Join

Suppose we are given two datasets: a labeled dataset and unlabeled datas...

Please sign up or login with your details

Forgot password? Click here to reset