FlowFace++: Explicit Semantic Flow-supervised End-to-End Face Swapping

by   Yu Zhang, et al.
Zhejiang University

This work proposes a novel face-swapping framework FlowFace++, utilizing explicit semantic flow supervision and end-to-end architecture to facilitate shape-aware face-swapping. Specifically, our work pretrains a facial shape discriminator to supervise the face swapping network. The discriminator is shape-aware and relies on a semantic flow-guided operation to explicitly calculate the shape discrepancies between the target and source faces, thus optimizing the face swapping network to generate highly realistic results. The face swapping network is a stack of a pre-trained face-masked autoencoder (MAE), a cross-attention fusion module, and a convolutional decoder. The MAE provides a fine-grained facial image representation space, which is unified for the target and source faces and thus facilitates final realistic results. The cross-attention fusion module carries out the source-to-target face swapping in a fine-grained latent space while preserving other attributes of the target image (e.g. expression, head pose, hair, background, illumination, etc). Lastly, the convolutional decoder further synthesizes the swapping results according to the face-swapping latent embedding from the cross-attention fusion module. Extensive quantitative and qualitative experiments on in-the-wild faces demonstrate that our FlowFace++ outperforms the state-of-the-art significantly, particularly while the source face is obstructed by uneven lighting or angle offset.


page 1

page 5

page 6

page 8

page 9

page 10

page 11

page 12


HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping

In this work, we propose a high fidelity face swapping method, called Hi...

Finding Directions in GAN's Latent Space for Neural Face Reenactment

This paper is on face/head reenactment where the goal is to transfer the...

End-to-end Face-swapping via Adaptive Latent Representation Learning

Taking full advantage of the excellent performance of StyleGAN, style tr...

One-shot Face Reenactment

To enable realistic shape (e.g. pose and expression) transfer, existing ...

StyO: Stylize Your Face in Only One-Shot

This paper focuses on face stylization with a single artistic target. Ex...

Instance-level Facial Attributes Transfer with Geometry-Aware Flow

We address the problem of instance-level facial attribute transfer witho...

3DFaceFill: An Analysis-By-Synthesis Approach to Face Completion

Existing face completion solutions are primarily driven by end-to-end mo...

Please sign up or login with your details

Forgot password? Click here to reset