DiffGAR: Model-Agnostic Restoration from Generative Artifacts Using Image-to-Image Diffusion Models

10/16/2022
by   Yueqin Yin, et al.
0

Recent generative models show impressive results in photo-realistic image generation. However, artifacts often inevitably appear in the generated results, leading to downgraded user experience and reduced performance in downstream tasks. This work aims to develop a plugin post-processing module for diverse generative models, which can faithfully restore images from diverse generative artifacts. This is challenging because: (1) Unlike traditional degradation patterns, generative artifacts are non-linear and the transformation function is highly complex. (2) There are no readily available artifact-image pairs. (3) Different from model-specific anti-artifact methods, a model-agnostic framework views the generator as a black-box machine and has no access to the architecture details. In this work, we first design a group of mechanisms to simulate generative artifacts of popular generators (i.e., GANs, autoregressive models, and diffusion models), given real images. Second, we implement the model-agnostic anti-artifact framework as an image-to-image diffusion model, due to its advantage in generation quality and capacity. Finally, we design a conditioning scheme for the diffusion model to enable both blind and non-blind image restoration. A guidance parameter is also introduced to allow for a trade-off between restoration accuracy and image quality. Extensive experiments show that our method significantly outperforms previous approaches on the proposed datasets and real-world artifact images.

READ FULL TEXT

page 1

page 6

page 11

08/29/2023

DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior

We present DiffBIR, which leverages pretrained text-to-image diffusion m...
04/03/2023

Generative Diffusion Prior for Unified Image Restoration and Enhancement

Existing image restoration methods mostly leverage the posterior distrib...
08/08/2021

BIGRoC: Boosting Image Generation via a Robust Classifier

The interest of the machine learning community in image synthesis has gr...
05/31/2023

A Unified Conditional Framework for Diffusion-based Image Restoration

Diffusion Probabilistic Models (DPMs) have recently shown remarkable per...
07/26/2023

Artifact Restoration in Histology Images with Diffusion Probabilistic Models

Histological whole slide images (WSIs) can be usually compromised by art...
09/06/2023

My Art My Choice: Adversarial Protection Against Unruly AI

Generative AI is on the rise, enabling everyone to produce realistic con...
09/19/2023

PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance

Exploiting pre-trained diffusion models for restoration has recently bec...

Please sign up or login with your details

Forgot password? Click here to reset