Gradually Updated Neural Networks for Large-Scale Image Recognition
share
We present a simple yet effective neural network architecture for image recognition. Unlike the previous state-of-the-art neural networks which usually have very deep architectures, we build networks that are shallower but can achieve better performances on three competitive benchmark datasets, i.e., CIFAR-10/100 and ImageNet. Our architectures are built using Gradually Updated Neural Network (GUNN) layers, which differ from the standard Convolutional Neural Network (CNN) layers in the way their output channels are computed: the CNN layers compute the output channels simultaneously while GUNN layers compute the channels gradually. By adding the computation ordering to the channels of CNNs, our networks are able to achieve better accuracies while using fewer layers and less memory. The architecture design of GUNN is guided by theoretical results and verified by empirical experiments. We set new records on the CIFAR-10 and CIFAR-100 datasets and achieve better accuracy on ImageNet under similar complexity with the previous state-of-the-art methods.
READ FULL TEXT