DeepAI AI Chat
Log In Sign Up

KL Guided Domain Adaptation

by   A. Tuan Nguyen, et al.

Domain adaptation is an important problem and often needed for real-world applications. In this problem, instead of i.i.d. datapoints, we assume that the source (training) data and the target (testing) data have different distributions. With that setting, the empirical risk minimization training procedure often does not perform well, since it does not account for the change in the distribution. A common approach in the domain adaptation literature is to learn a representation of the input that has the same distributions over the source and the target domain. However, these approaches often require additional networks and/or optimizing an adversarial (minimax) objective, which can be very expensive or unstable in practice. To tackle this problem, we first derive a generalization bound for the target loss based on the training loss and the reverse Kullback-Leibler (KL) divergence between the source and the target representation distributions. Based on this bound, we derive an algorithm that minimizes the KL term to obtain a better generalization to the target domain. We show that with a probabilistic representation network, the KL term can be estimated efficiently via minibatch samples without any additional network or a minimax objective. This leads to a theoretically sound alignment method which is also very efficient and stable in practice. Experimental results also suggest that our method outperforms other representation-alignment approaches.


page 1

page 2

page 3

page 4


Learning Bounds for Moment-Based Domain Adaptation

Domain adaptation algorithms are designed to minimize the misclassificat...

Distributionally Robust Domain Adaptation

Domain Adaptation (DA) has recently received significant attention due t...

Domain Adaptation via Rebalanced Sub-domain Alignment

Unsupervised domain adaptation (UDA) is a technique used to transfer kno...

Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment

Domain adaptation addresses the common problem when the target distribut...

Efficient Representation of Large-Alphabet Probability Distributions

A number of engineering and scientific problems require representing and...

Collaborative Training of Balanced Random Forests for Open Set Domain Adaptation

In this paper, we introduce a collaborative training algorithm of balanc...