A Distillation Learning Model of Adaptive Structural Deep Belief Network for AffectNet: Facial Expression Image Database

by   Takumi Ichimura, et al.
Prefectural University of Hiroshima

Deep Learning has a hierarchical network architecture to represent the complicated feature of input patterns. We have developed the adaptive structure learning method of Deep Belief Network (DBN) that can discover an optimal number of hidden neurons for given input data in a Restricted Boltzmann Machine (RBM) by neuron generation-annihilation algorithm, and can obtain the appropriate number of hidden layers in DBN. In this paper, our model is applied to a facial expression image data set, AffectNet. The system has higher classification capability than the traditional CNN. However, our model was not able to classify some test cases correctly because human emotions contain many ambiguous features or patterns leading wrong answer by two or more annotators who have different subjective judgment for a facial image. In order to represent such cases, this paper investigated a distillation learning model of Adaptive DBN. The original trained model can be seen as a parent model and some child models are trained for some mis-classified cases. For the difference between the parent model and the child one, KL divergence is monitored and then some appropriate new neurons at the parent model are generated according to KL divergence to improve classification accuracy. In this paper, the classification accuracy was improved from 78.4 method.


page 1

page 3


Adaptive Learning Method of Recurrent Temporal Deep Belief Network to Analyze Time Series Data

Deep Learning has the hierarchical network architecture to represent the...

An Adaptive Learning Method of Deep Belief Network by Layer Generation Algorithm

Deep Belief Network (DBN) has a deep architecture that represents multip...

An Object Detection by using Adaptive Structural Learning of Deep Belief Network

Deep learning forms a hierarchical network structure for representation ...

Please sign up or login with your details

Forgot password? Click here to reset