MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN

by   Wenlong Cheng, et al.

Generative adversarial networks (GANs) have promoted remarkable advances in single-image super-resolution (SR) by recovering photo-realistic images. However, high memory consumption of GAN-based SR (usually generators) causes performance degradation and more energy consumption, hindering the deployment of GAN-based SR into resource-constricted mobile devices. In this paper, we propose a novel compression framework Multi-scale Feature Aggregation Net based GAN (MFAGAN) for reducing the memory access cost of the generator. First, to overcome the memory explosion of dense connections, we utilize a memory-efficient multi-scale feature aggregation net as the generator. Second, for faster and more stable training, our method introduces the PatchGAN discriminator. Third, to balance the student discriminator and the compressed generator, we distill both the generator and the discriminator. Finally, we perform a hardware-aware neural architecture search (NAS) to find a specialized SubGenerator for the target mobile phone. Benefiting from these improvements, the proposed MFAGAN achieves up to 8.3× memory saving and 42.9× computation reduction, with only minor visual quality degradation, compared with ESRGAN. Empirical studies also show ∼70 milliseconds latency on Qualcomm Snapdragon 865 chipset.


page 1

page 5

page 8


Journey Towards Tiny Perceptual Super-Resolution

Recent works in single-image perceptual super-resolution (SR) have demon...

A-ESRGAN: Training Real-World Blind Super-Resolution with Attention U-Net Discriminators

Blind image super-resolution(SR) is a long-standing task in CV that aims...

Fine-grained Attention and Feature-sharing Generative Adversarial Networks for Single Image Super-Resolution

The traditional super-resolution methods that aim to minimize the mean s...

Multi-scale deep neural networks for real image super-resolution

Single image super-resolution (SR) is extremely difficult if the upscali...

GAN Vocoder: Multi-Resolution Discriminator Is All You Need

Several of the latest GAN-based vocoders show remarkable achievements, o...

Latent Multi-Relation Reasoning for GAN-Prior based Image Super-Resolution

Recently, single image super-resolution (SR) under large scaling factors...

A Three-Player GAN for Super-Resolution in Magnetic Resonance Imaging

Learning based single image super resolution (SISR) task is well investi...

Please sign up or login with your details

Forgot password? Click here to reset