Supervising Unsupervised Learning with Evolutionary Algorithm in Deep Neural Network

by   Takeshi Inagaki, et al.

A method to control results of gradient descent unsupervised learning in a deep neural network by using evolutionary algorithm is proposed. To process crossover of unsupervisedly trained models, the algorithm evaluates pointwise fitness of individual nodes in neural network. Labeled training data is randomly sampled and breeding process selects nodes by calculating degree of their consistency on different sets of sampled data. This method supervises unsupervised training by evolutionary process. We also introduce modified Restricted Boltzmann Machine which contains repulsive force among nodes in a neural network and it contributes to isolate network nodes each other to avoid accidental degeneration of nodes by evolutionary process. These new methods are applied to document classification problem and it results better accuracy than a traditional fully supervised classifier implemented with linear regression algorithm.


page 1

page 2

page 3

page 4


Learning to Generate Genotypes with Neural Networks

Neural networks and evolutionary computation have a rich intertwined his...

Sampled Training and Node Inheritance for Fast Evolutionary Neural Architecture Search

The performance of a deep neural network is heavily dependent on its arc...

GADAM: Genetic-Evolutionary ADAM for Deep Neural Network Optimization

Deep neural network learning can be formulated as a non-convex optimizat...

ENSURE: Ensemble Stein's Unbiased Risk Estimator for Unsupervised Learning

Deep learning accelerates the MR image reconstruction process after offl...

Active Learning for Node Classification: The Additional Learning Ability from Unlabelled Nodes

Node classification on graph data is an important task on many practical...

An Evolutionary Algorithm of Linear complexity: Application to Training of Deep Neural Networks

The performance of deep neural networks, such as Deep Belief Networks fo...

Self Organizing Classifiers: First Steps in Structured Evolutionary Machine Learning

Learning classifier systems (LCSs) are evolutionary machine learning alg...

Please sign up or login with your details

Forgot password? Click here to reset