Fine-Grained Face Swapping via Regional GAN Inversion

by   Zhian Liu, et al.

We present a novel paradigm for high-fidelity face swapping that faithfully preserves the desired subtle geometry and texture details. We rethink face swapping from the perspective of fine-grained face editing, i.e., “editing for swapping” (E4S), and propose a framework that is based on the explicit disentanglement of the shape and texture of facial components. Following the E4S principle, our framework enables both global and local swapping of facial features, as well as controlling the amount of partial swapping specified by the user. Furthermore, the E4S paradigm is inherently capable of handling facial occlusions by means of facial masks. At the core of our system lies a novel Regional GAN Inversion (RGI) method, which allows the explicit disentanglement of shape and texture. It also allows face swapping to be performed in the latent space of StyleGAN. Specifically, we design a multi-scale mask-guided encoder to project the texture of each facial component into regional style codes. We also design a mask-guided injection module to manipulate the feature maps with the style codes. Based on the disentanglement, face swapping is reformulated as a simplified problem of style and mask swapping. Extensive experiments and comparisons with current state-of-the-art methods demonstrate the superiority of our approach in preserving texture and shape details, as well as working with high resolution images at 1024×1024.


page 1

page 4

page 6

page 7

page 8


Designing a 3D-Aware StyleNeRF Encoder for Face Editing

GAN inversion has been exploited in many face manipulation tasks, but 2D...

Text-guided Eyeglasses Manipulation with Spatial Constraints

Virtual try-on of eyeglasses involves placing eyeglasses of different sh...

Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval

We present Retrieve in Style (RIS), an unsupervised framework for fine-g...

FENeRF: Face Editing in Neural Radiance Fields

Previous portrait image generation methods roughly fall into two categor...

Towards High Fidelity Face Relighting with Realistic Shadows

Existing face relighting methods often struggle with two problems: maint...

MaskGAN: Towards Diverse and Interactive Facial Image Manipulation

Facial image manipulation has achieved great progresses in recent years....

Learning Oracle Attention for High-fidelity Face Completion

High-fidelity face completion is a challenging task due to the rich and ...

Please sign up or login with your details

Forgot password? Click here to reset