Unsupervised Domain Adaptation: A Reality Check

by   Kevin Musgrave, et al.

Interest in unsupervised domain adaptation (UDA) has surged in recent years, resulting in a plethora of new algorithms. However, as is often the case in fast-moving fields, baseline algorithms are not tested to the extent that they should be. Furthermore, little attention has been paid to validation methods, i.e. the methods for estimating the accuracy of a model in the absence of target domain labels. This is despite the fact that validation methods are a crucial component of any UDA train/val pipeline. In this paper, we show via large-scale experimentation that 1) in the oracle setting, the difference in accuracy between UDA algorithms is smaller than previously thought, 2) state-of-the-art validation methods are not well-correlated with accuracy, and 3) differences between UDA algorithms are dwarfed by the drop in accuracy caused by validation methods.


page 5

page 6

page 8

page 13

page 15

page 16

page 19

page 20


Benchmarking Validation Methods for Unsupervised Domain Adaptation

This paper compares and ranks 11 UDA validation methods. Validators esti...

Model Selection with Nonlinear Embedding for Unsupervised Domain Adaptation

Domain adaptation deals with adapting classifiers trained on data from a...

Noise transfer for unsupervised domain adaptation of retinal OCT images

Optical coherence tomography (OCT) imaging from different camera devices...

ADL-ID: Adversarial Disentanglement Learning for Wireless Device Fingerprinting Temporal Domain Adaptation

As the journey of 5G standardization is coming to an end, academia and i...

Knothe-Rosenblatt transport for Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims at exploiting related but diff...

Addressing Parameter Choice Issues in Unsupervised Domain Adaptation by Aggregation

We study the problem of choosing algorithm hyper-parameters in unsupervi...

Better Practices for Domain Adaptation

Distribution shifts are all too common in real-world applications of mac...

Please sign up or login with your details

Forgot password? Click here to reset