Weakly Supervised Correspondence Learning

03/02/2022
by   Zihan Wang, et al.
20

Correspondence learning is a fundamental problem in robotics, which aims to learn a mapping between state, action pairs of agents of different dynamics or embodiments. However, current correspondence learning methods either leverage strictly paired data – which are often difficult to collect – or learn in an unsupervised fashion from unpaired data using regularization techniques such as cycle-consistency – which suffer from severe misalignment issues. We propose a weakly supervised correspondence learning approach that trades off between strong supervision over strictly paired data and unsupervised learning with a regularizer over unpaired data. Our idea is to leverage two types of weak supervision: i) temporal ordering of states and actions to reduce the compounding error, and ii) paired abstractions, instead of paired data, to alleviate the misalignment problem and learn a more accurate correspondence. The two types of weak supervision are easy to access in real-world applications, which simultaneously reduces the high cost of annotating strictly paired data and improves the quality of the learned correspondence.

READ FULL TEXT

page 1

page 6

research
06/03/2019

An Adaptable Framework for Deep Adversarial Label Learning from Weak Supervision

In this paper, we propose a general framework for using adversarial labe...
research
01/09/2023

Learning Bidirectional Action-Language Translation with Limited Supervision and Incongruent Input

Human infant learning happens during exploration of the environment, by ...
research
08/14/2021

Weakly Supervised Continual Learning

Continual Learning (CL) investigates how to train Deep Networks on a str...
research
12/13/2019

Action Modifiers: Learning from Adverbs in Instructional Videos

We present a method to learn a representation for adverbs from instructi...
research
08/20/2019

Phrase Localization Without Paired Training Examples

Localizing phrases in images is an important part of image understanding...
research
01/09/2019

D^3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentation

We address weakly-supervised action alignment and segmentation in videos...
research
05/29/2020

Overview of Scanner Invariant Representations

Pooled imaging data from multiple sources is subject to bias from each s...

Please sign up or login with your details

Forgot password? Click here to reset