Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation

03/12/2015

∙

Compared to Multilayer Neural Networks with real weights, Binary Multilayer Neural Networks (BMNNs) can be implemented more efficiently on dedicated hardware. BMNNs have been demonstrated to be effective on binary classification tasks with Expectation BackPropagation (EBP) algorithm on high dimensional text datasets. In this paper, we investigate the capability of BMNNs using the EBP algorithm on multiclass image classification tasks. The performances of binary neural networks with multiple hidden layers and different numbers of hidden units are examined on MNIST. We also explore the effectiveness of image spatial filters and the dropout technique in BMNNs. Experimental results on MNIST dataset show that EBP can obtain 2.12 test error with real weights, which is comparable to the results of standard BackPropagation algorithm on fully connected MNNs.

READ FULL TEXT

Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation

Mean Field Bayes Backpropagation: scalable training of multilayer neural networks with binary weights

Chi-Square Test Neural Network: A New Binary Classifier based on Backpropagation Neural Network

DropFilter: A Novel Regularization Method for Learning Convolutional Neural Networks

KCNet: An Insect-Inspired Single-Hidden-Layer Neural Network with Randomized Binary Weights for Prediction and Classification Tasks

Structured DropConnect for Uncertainty Inference in Image Classification

Avoiding overfitting of multilayer perceptrons by training derivatives

Multilayer Lookahead: a Nested Version of Lookahead

Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation

Related Research

Mean Field Bayes Backpropagation: scalable training of multilayer neural networks with binary weights

Chi-Square Test Neural Network: A New Binary Classifier based on Backpropagation Neural Network

DropFilter: A Novel Regularization Method for Learning Convolutional Neural Networks

KCNet: An Insect-Inspired Single-Hidden-Layer Neural Network with Randomized Binary Weights for Prediction and Classification Tasks

Structured DropConnect for Uncertainty Inference in Image Classification

Avoiding overfitting of multilayer perceptrons by training derivatives

Multilayer Lookahead: a Nested Version of Lookahead