A general approach to bridge the reality-gap

by   Michael Lomnitz, et al.

Employing machine learning models in the real world requires collecting large amounts of data, which is both time consuming and costly to collect. A common approach to circumvent this is to leverage existing, similar data-sets with large amounts of labelled data. However, models trained on these canonical distributions do not readily transfer to real-world ones. Domain adaptation and transfer learning are often used to breach this "reality gap", though both require a substantial amount of real-world data. In this paper we discuss a more general approach: we propose learning a general transformation to bring arbitrary images towards a canonical distribution where we can naively apply the trained machine learning models. This transformation is trained in an unsupervised regime, leveraging data augmentation to generate off-canonical examples of images and training a Deep Learning model to recover their original counterpart. We quantify the performance of this transformation using pre-trained ImageNet classifiers, demonstrating that this procedure can recover half of the loss in performance on the distorted data-set. We then validate the effectiveness of this approach on a series of pre-trained ImageNet models on a real world data set collected by printing and photographing images in different lighting conditions.


page 6

page 7


Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks

Real world data, especially in the domain of robotics, is notoriously co...

Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trails

Robots hold promise in many scenarios involving outdoor use, such as sea...

A novel algorithm can generate data to train machine learning models in conditions of extreme scarcity of real world data

Training machine learning models requires large datasets. However, colle...

Does progress on ImageNet transfer to real-world datasets?

Does progress on ImageNet transfer to real-world datasets? We investigat...

Towards a theory of out-of-distribution learning

What is learning? 20^st century formalizations of learning theory – whic...

Representational Multiplicity Should Be Exposed, Not Eliminated

It is prevalent and well-observed, but poorly understood, that two machi...

Data Provenance via Differential Auditing

Auditing Data Provenance (ADP), i.e., auditing if a certain piece of dat...

Please sign up or login with your details

Forgot password? Click here to reset