Contrastive Variational Autoencoder Enhances Salient Features

by   Abubakar Abid, et al.

Variational autoencoders are powerful algorithms for identifying dominant latent structure in a single dataset. In many applications, however, we are interested in modeling latent structure and variation that are enriched in a target dataset compared to some background---e.g. enriched in patients compared to the general population. Contrastive learning is a principled framework to capture such enriched variation between the target and background, but state-of-the-art contrastive methods are limited to linear models. In this paper, we introduce the contrastive variational autoencoder (cVAE), which combines the benefits of contrastive learning with the power of deep generative models. The cVAE is designed to identify and enhance salient latent features. The cVAE is trained on two related but unpaired datasets, one of which has minimal contribution from the salient latent features. The cVAE explicitly models latent features that are shared between the datasets, as well as those that are enriched in one dataset relative to the other, which allows the algorithm to isolate and enhance the salient latent features. The algorithm is straightforward to implement, has a similar run-time to the standard VAE, and is robust to noise and dataset purity. We conduct experiments across diverse types of data, including gene expression and facial images, showing that the cVAE effectively uncovers latent structure that is salient in a particular analysis.


page 2

page 5

page 7

page 8

page 10

page 11

page 14


SepVAE: a contrastive VAE to separate pathological patterns from healthy ones

Contrastive Analysis VAE (CA-VAEs) is a family of Variational auto-encod...

Moment Matching Deep Contrastive Latent Variable Models

In the contrastive analysis (CA) setting, machine learning practitioners...

Noise Contrastive Variational Autoencoders

We take steps towards understanding the "posterior collapse (PC)" diffic...

Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation

Sequential recommendation as an emerging topic has attracted increasing ...

RGCVAE: Relational Graph Conditioned Variational Autoencoder for Molecule Design

Identifying molecules that exhibit some pre-specified properties is a di...

Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification

Multi-label classification (MLC) is a prediction task where each sample ...

Generating private data with user customization

Personal devices such as mobile phones can produce and store large amoun...

Please sign up or login with your details

Forgot password? Click here to reset