Autoencoding Under Normalization Constraints

by   Sangwoong Yoon, et al.

Likelihood is a standard estimate for outlier detection. The specific role of the normalization constraint is to ensure that the out-of-distribution (OOD) regime has a small likelihood when samples are learned using maximum likelihood. Because autoencoders do not possess such a process of normalization, they often fail to recognize outliers even when they are obviously OOD. We propose the Normalized Autoencoder (NAE), a normalized probabilistic model constructed from an autoencoder. The probability density of NAE is defined using the reconstruction error of an autoencoder, which is differently defined in the conventional energy-based model. In our model, normalization is enforced by suppressing the reconstruction of negative samples, significantly improving the outlier detection performance. Our experimental results confirm the efficacy of NAE, both in detecting outliers and in generating in-distribution samples.


page 1

page 7

page 8

page 13

page 14


Statistical Depth based Normalization and Outlier Detection of Gene Expression Data

Normalization and outlier detection belong to the preprocessing of gene ...

Probabilistic Outlier Detection and Generation

A new method for outlier detection and generation is introduced by lifti...

Unsupervised Out-of-Distribution Detection with Batch Normalization

Likelihood from a generative model is a natural statistic for detecting ...

The Importance of Suppressing Complete Reconstruction in Autoencoders for Unsupervised Outlier Detection

Autoencoders are widely used in outlier detection due to their superiori...

Negative Sampling in Variational Autoencoders

We propose negative sampling as an approach to improve the notoriously b...

Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection

After an autoencoder (AE) has learnt to reconstruct one dataset, it migh...

Code Repositories


The official repository for (Yoon, Noh and Park, ICML 2021).

view repo

Please sign up or login with your details

Forgot password? Click here to reset