Sparsifying Binary Networks

by   Riccardo Schiavone, et al.

Binary neural networks (BNNs) have demonstrated their ability to solve complex tasks with comparable accuracy as full-precision deep neural networks (DNNs), while also reducing computational power and storage requirements and increasing the processing speed. These properties make them an attractive alternative for the development and deployment of DNN-based applications in Internet-of-Things (IoT) devices. Despite the recent improvements, they suffer from a fixed and limited compression factor that may result insufficient for certain devices with very limited resources. In this work, we propose sparse binary neural networks (SBNNs), a novel model and training scheme which introduces sparsity in BNNs and a new quantization function for binarizing the network's weights. The proposed SBNN is able to achieve high compression factors and it reduces the number of operations and parameters at inference time. We also provide tools to assist the SBNN design, while respecting hardware resource constraints. We study the generalization properties of our method for different compression factors through a set of experiments on linear and convolutional networks on three datasets. Our experiments confirm that SBNNs can achieve high compression rates, without compromising generalization, while further reducing the operations of BNNs, making SBNNs a viable option for deploying DNNs in cheap, low-cost, limited-resources IoT devices and sensors.


page 1

page 2

page 3

page 4


Compacting Deep Neural Networks for Internet of Things: Methods and Applications

Deep Neural Networks (DNNs) have shown great success in completing compl...

Binary domain generalization for sparsifying binary neural networks

Binary neural networks (BNNs) are an attractive solution for developing ...

Deploy Large-Scale Deep Neural Networks in Resource Constrained IoT Devices with Local Quantization Region

Implementing large-scale deep neural networks with high computational co...

Neural Network Compression Via Sparse Optimization

The compression of deep neural networks (DNNs) to reduce inference cost ...

A Survey on Deep Neural Network Compression: Challenges, Overview, and Solutions

Deep Neural Network (DNN) has gained unprecedented performance due to it...

Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Rerouting

Deep neural networks (DNNs) have shown to provide superb performance in ...

InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks

The promise of Deep Neural Network (DNN) powered Internet of Thing (IoT)...

Please sign up or login with your details

Forgot password? Click here to reset