Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression

by   Yawei Li, et al.
ETH Zurich

In this paper, we analyze two popular network compression techniques, i.e. filter pruning and low-rank decomposition, in a unified sense. By simply changing the way the sparsity regularization is enforced, filter pruning and low-rank decomposition can be derived accordingly. This provides another flexible choice for network compression because the techniques complement each other. For example, in popular network architectures with shortcut connections (e.g. ResNet), filter pruning cannot deal with the last convolutional layer in a ResBlock while the low-rank decomposition methods can. In addition, we propose to compress the whole network jointly instead of in a layer-wise manner. Our approach proves its potential as it compares favorably to the state-of-the-art on several benchmarks.


page 13

page 14


Low-Rank+Sparse Tensor Compression for Neural Networks

Low-rank tensor compression has been proposed as a promising approach to...

TRP: Trained Rank Pruning for Efficient Deep Neural Networks

To enable DNNs on edge devices like mobile phones, low-rank approximatio...

Traned Rank Pruning for Efficient Deep Neural Networks

To accelerate DNNs inference, low-rank approximation has been widely ado...

Cascaded Projection: End-to-End Network Compression and Acceleration

We propose a data-driven approach for deep convolutional neural network ...

Vision Transformer Compression with Structured Pruning and Low Rank Approximation

Transformer architecture has gained popularity due to its ability to sca...

Entropy Induced Pruning Framework for Convolutional Neural Networks

Structured pruning techniques have achieved great compression performanc...

Towards Compact CNNs via Collaborative Compression

Channel pruning and tensor decomposition have received extensive attenti...

Code Repositories


Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression. CVPR2020.

view repo

Please sign up or login with your details

Forgot password? Click here to reset