Provable Filter Pruning for Efficient Neural Networks

11/18/2019
by   Lucas Liebenwein, et al.
15

We present a provable, sampling-based approach for generating compact Convolutional Neural Networks (CNNs) by identifying and removing redundant filters from an over-parameterized network. Our algorithm uses a small batch of input data points to assign a saliency score to each filter and constructs an importance sampling distribution where filters that highly affect the output are sampled with correspondingly high probability. In contrast to existing filter pruning approaches, our method is simultaneously data-informed, exhibits provable guarantees on the size and performance of the pruned network, and is widely applicable to varying network architectures and data sets. Our analytical bounds bridge the notions of compressibility and importance of network structures, which gives rise to a fully-automated procedure for identifying and preserving filters in layers that are essential to the network's performance. Our experimental evaluations on popular architectures and data sets show that our algorithm consistently generates sparser and more efficient models than those constructed by existing filter pruning approaches.

READ FULL TEXT
research
10/11/2019

SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks

We introduce a pruning algorithm that provably sparsifies the parameters...
research
04/05/2023

Efficient CNNs via Passive Filter Pruning

Convolutional neural networks (CNNs) have shown state-of-the-art perform...
research
08/08/2023

D-Score: A Synapse-Inspired Approach for Filter Pruning

This paper introduces a new aspect for determining the rank of the unimp...
research
10/25/2022

Toward domain generalized pruning by scoring out-of-distribution importance

Filter pruning has been widely used for compressing convolutional neural...
research
05/06/2020

Dependency Aware Filter Pruning

Convolutional neural networks (CNNs) are typically over-parameterized, b...
research
12/07/2021

i-SpaSP: Structured Neural Pruning via Sparse Signal Recovery

We propose a novel, structured pruning algorithm for neural networks – t...
research
03/05/2020

Cluster Pruning: An Efficient Filter Pruning Method for Edge AI Vision Applications

Even though the Convolutional Neural Networks (CNN) has shown superior r...

Please sign up or login with your details

Forgot password? Click here to reset