A Hybrid of Generative and Discriminative Models Based on the Gaussian-coupled Softmax Layer

by   Hideaki Hayashi, et al.
Osaka University

Generative models have advantageous characteristics for classification tasks such as the availability of unsupervised data and calibrated confidence, whereas discriminative models have advantages in terms of the simplicity of their model structures and learning algorithms and their ability to outperform their generative counterparts. In this paper, we propose a method to train a hybrid of discriminative and generative models in a single neural network (NN), which exhibits the characteristics of both models. The key idea is the Gaussian-coupled softmax layer, which is a fully connected layer with a softmax activation function coupled with Gaussian distributions. This layer can be embedded into an NN-based classifier and allows the classifier to estimate both the class posterior distribution and the class-conditional data distribution. We demonstrate that the proposed hybrid model can be applied to semi-supervised learning and confidence calibration.


page 1

page 2

page 6

page 7

page 8

page 9


SDGM: Sparse Bayesian Classifier Based on a Discriminative Gaussian Mixture Model

In probabilistic classification, a discriminative model based on Gaussia...

Neural Network Representation Control: Gaussian Isolation Machines and CVC Regularization

In many cases, neural network classifiers are likely to be exposed to in...

Money on the Table: Statistical information ignored by Softmax can improve classifier accuracy

Softmax is a standard final layer used in Neural Nets (NNs) to summarize...

Generative Robust Classification

Training adversarially robust discriminative (i.e., softmax) classifier ...

Generative and Discriminative Text Classification with Recurrent Neural Networks

We empirically characterize the performance of discriminative and genera...

On The Distribution of Penultimate Activations of Classification Networks

This paper studies probability distributions of penultimate activations ...

The Projected Belief Network Classfier : both Generative and Discriminative

The projected belief network (PBN) is a layered generative network with ...

Please sign up or login with your details

Forgot password? Click here to reset