FALCON: Fast and Lightweight Convolution for Compressing and Accelerating CNN

by   Chun Quan, et al.

How can we efficiently compress Convolutional Neural Networks (CNN) while retaining their accuracy on classification tasks? A promising direction is based on depthwise separable convolution which replaces a standard convolution with a depthwise convolution and a pointwise convolution. However, previous works based on depthwise separable convolution are limited since 1) they are mostly heuristic approaches without a precise understanding of their relations to standard convolution, and 2) their accuracies do not match that of the standard convolution. In this paper, we propose FALCON, an accurate and lightweight method for compressing CNN. FALCON is derived by interpreting existing convolution methods based on depthwise separable convolution using EHP, our proposed mathematical formulation to approximate the standard convolution kernel. Such interpretation leads to developing a generalized version rank-k FALCON which further improves the accuracy while sacrificing a bit of compression and computation reduction rates. In addition, we propose FALCON-branch by fitting FALCON into the previous state-of-the-art convolution unit ShuffleUnitV2 which gives even better accuracy. Experiments show that FALCON and FALCON-branch outperform 1) existing methods based on depthwise separable convolution and 2) standard CNN models by up to 8x compression and 8x computation reduction while ensuring similar accuracy. We also demonstrate that rank-k FALCON provides even better accuracy than standard convolution in many cases, while using a smaller number of parameters and floating-point operations.


page 1

page 2

page 3

page 4


Compression of convolutional neural networks for high performance imagematching tasks on mobile devices

Deep neural networks have demonstrated state-of-the-art performance for ...

3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks

Standard 3D convolution operations require much larger amounts of memory...

Learning Graph Convolution Filters from Data Manifold

Convolution Neural Network (CNN) has gained tremendous success in comput...

Depthwise Multiception Convolution for Reducing Network Parameters without Sacrificing Accuracy

Deep convolutional neural networks have been proven successful in multip...

A Quantization-Friendly Separable Convolution for MobileNets

As deep learning (DL) is being rapidly pushed to edge computing, researc...

PENNI: Pruned Kernel Sharing for Efficient CNN Inference

Although state-of-the-art (SOTA) CNNs achieve outstanding performance on...

Fast Dynamic Convolutional Neural Networks for Visual Tracking

Most of the existing tracking methods based on CNN(convolutional neural ...

Please sign up or login with your details

Forgot password? Click here to reset