DeepAI AI Chat
Log In Sign Up

Contrastive Representation Learning with Trainable Augmentation Channel

11/15/2021
by   Masanori Koyama, et al.
University of Oxford
Preferred Infrastructure
0

In contrastive representation learning, data representation is trained so that it can classify the image instances even when the images are altered by augmentations. However, depending on the datasets, some augmentations can damage the information of the images beyond recognition, and such augmentations can result in collapsed representations. We present a partial solution to this problem by formalizing a stochastic encoding process in which there exist a tug-of-war between the data corruption introduced by the augmentations and the information preserved by the encoder. We show that, with the infoMax objective based on this framework, we can learn a data-dependent distribution of augmentations to avoid the collapse of the representation.

READ FULL TEXT

page 2

page 4

02/05/2023

CIPER: Combining Invariant and Equivariant Representations Using Contrastive and Predictive Learning

Self-supervised representation learning (SSRL) methods have shown great ...
07/05/2022

Features Based Adaptive Augmentation for Graph Contrastive Learning

Self-Supervised learning aims to eliminate the need for expensive annota...
11/05/2020

Center-wise Local Image Mixture For Contrastive Representation Learning

Recent advances in unsupervised representation learning have experienced...
10/05/2020

A Simple Framework for Uncertainty in Contrastive Learning

Contrastive approaches to representation learning have recently shown gr...
02/03/2023

Contrastive Learning with Consistent Representations

Contrastive learning demonstrates great promise for representation learn...
01/15/2023

MN-Pair Contrastive Damage Representation and Clustering for Prognostic Explanation

It is essential for infrastructure managers to maintain a high standard ...
10/19/2022

UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning

Detecting out-of-domain (OOD) intents from user queries is essential for...