Stitchable Neural Networks

by   Zizheng Pan, et al.

The public model zoo containing enormous powerful pretrained model families (e.g., ResNet/DeiT) has reached an unprecedented scope than ever, which significantly contributes to the success of deep learning. As each model family consists of pretrained models with diverse scales (e.g., DeiT-Ti/S/B), it naturally arises a fundamental question of how to efficiently assemble these readily available models in a family for dynamic accuracy-efficiency trade-offs at runtime. To this end, we present Stitchable Neural Networks (SN-Net), a novel scalable and efficient framework for model deployment which cheaply produces numerous networks with different complexity and performance trade-offs given a family of pretrained neural networks, which we call anchors. Specifically, SN-Net splits the anchors across the blocks/layers and then stitches them together with simple stitching layers to map the activations from one anchor to another. With only a few epochs of training, SN-Net effectively interpolates between the performance of anchors with varying scales. At runtime, SN-Net can instantly adapt to dynamic resource constraints by switching the stitching positions. Extensive experiments on ImageNet classification demonstrate that SN-Net can obtain on-par or even better performance than many individually trained networks while supporting diverse deployment scenarios. For example, by stitching Swin Transformers, we challenge hundreds of models in Timm model zoo with a single network. We believe this new elastic model framework can serve as a strong baseline for further research in wider communities.


page 1

page 2

page 3

page 4


Stitched ViTs are Flexible Vision Backbones

Large pretrained plain vision Transformers (ViTs) have been the workhors...

Any-Precision Deep Neural Networks

We present Any-Precision Deep Neural Networks (Any-Precision DNNs), whic...

Slimmable Neural Networks

We present a simple and general method to train a single neural network ...

Learning to Learn Parameterized Classification Networks for Scalable Input Images

Convolutional Neural Networks (CNNs) do not have a predictable recogniti...

Fully Dynamic Inference with Deep Neural Networks

Modern deep neural networks are powerful and widely applicable models th...

Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting

Although deep learning has made strides in the field of deep noise suppr...

Bespoke: A Block-Level Neural Network Optimization Framework for Low-Cost Deployment

As deep learning models become popular, there is a lot of need for deplo...

Please sign up or login with your details

Forgot password? Click here to reset