Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion

by   Mingbao Lin, et al.

The mainstream approach for filter pruning is usually either to force a hard-coded importance estimation upon a computation-heavy pretrained model to select "important" filters, or to impose a hyperparameter-sensitive sparse constraint on the loss objective to regularize the network training. In this paper, we present a novel filter pruning method, dubbed dynamic-coded filter fusion (DCFF), to derive compact CNNs in a computation-economical and regularization-free manner for efficient image classification. Each filter in our DCFF is firstly given an inter-similarity distribution with a temperature parameter as a filter proxy, on top of which, a fresh Kullback-Leibler divergence based dynamic-coded criterion is proposed to evaluate the filter importance. In contrast to simply keeping high-score filters in other methods, we propose the concept of filter fusion, i.e., the weighted averages using the assigned proxies, as our preserved filters. We obtain a one-hot inter-similarity distribution as the temperature parameter approaches infinity. Thus, the relative importance of each filter can vary along with the training of the compact CNN, leading to dynamically changeable fused filters without both the dependency on the pretrained model and the introduction of sparse constraints. Extensive experiments on classification benchmarks demonstrate the superiority of our DCFF over the compared counterparts. For example, our DCFF derives a compact VGGNet-16 with only 72.77M FLOPs and 1.06M parameters while reaching top-1 accuracy of 93.47 with 63.8 accuracy on ILSVRC-2012. Our code, narrower models and training logs are available at


page 1

page 9


Data Agnostic Filter Gating for Efficient Deep Networks

To deploy a well-trained CNN model on low-end computation edge devices, ...

Meta Filter Pruning to Accelerate Deep Convolutional Neural Networks

Existing methods usually utilize pre-defined criterions, such as p-norm,...

Filter Pruning using Hierarchical Group Sparse Regularization for Deep Convolutional Neural Networks

Since the convolutional neural networks are often trained with redundant...

Online Filter Clustering and Pruning for Efficient Convnets

Pruning filters is an effective method for accelerating deep neural netw...

Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler

Filter pruning simultaneously accelerates the computation and reduces th...

Adaptive aggregation of Monte Carlo augmented decomposed filters for efficient group-equivariant convolutional neural network

Filter-decomposition-based group-equivariant convolutional neural networ...

Evolutionary design of photometric systems and its application to Gaia

Designing a photometric system to best fulfil a set of scientific goals ...

Please sign up or login with your details

Forgot password? Click here to reset