How far can we go without convolution: Improving fully-connected networks

11/09/2015
by   Zhouhan Lin, et al.
0

We propose ways to improve the performance of fully connected networks. We found that two approaches in particular have a strong effect on performance: linear bottleneck layers and unsupervised pre-training using autoencoders without hidden unit biases. We show how both approaches can be related to improving gradient flow and reducing sparsity in the network. We show that a fully connected network can yield approximately 70 the permutation-invariant CIFAR-10 task, which is much higher than the current state-of-the-art. By adding deformations to the training data, the fully connected network achieves 78 convolutional network.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset