Designing a 3D-Aware StyleNeRF Encoder for Face Editing

by   Songlin Yang, et al.

GAN inversion has been exploited in many face manipulation tasks, but 2D GANs often fail to generate multi-view 3D consistent images. The encoders designed for 2D GANs are not able to provide sufficient 3D information for the inversion and editing. Therefore, 3D-aware GAN inversion is proposed to increase the 3D editing capability of GANs. However, the 3D-aware GAN inversion remains under-explored. To tackle this problem, we propose a 3D-aware (3Da) encoder for GAN inversion and face editing based on the powerful StyleNeRF model. Our proposed 3Da encoder combines a parametric 3D face model with a learnable detail representation model to generate geometry, texture and view direction codes. For more flexible face manipulation, we then design a dual-branch StyleFlow module to transfer the StyleNeRF codes with disentangled geometry and texture flows. Extensive experiments demonstrate that we realize 3D consistent face manipulation in both facial attribute editing and texture transfer. Furthermore, for video editing, we make the sequence of frame codes share a common canonical manifold, which improves the temporal consistency of the edited attributes.


page 1

page 2

page 3

page 4


FENeRF: Face Editing in Neural Radiance Fields

Previous portrait image generation methods roughly fall into two categor...

IDE-3D: Interactive Disentangled Editing for High-Resolution 3D-aware Portrait Synthesis

Existing 3D-aware facial generation methods face a dilemma in quality ve...

Fine-Grained Face Swapping via Regional GAN Inversion

We present a novel paradigm for high-fidelity face swapping that faithfu...

3DAvatarGAN: Bridging Domains for Personalized Editable Avatars

Modern 3D-GANs synthesize geometry and texture by training on large-scal...

NARRATE: A Normal Assisted Free-View Portrait Stylizer

In this work, we propose NARRATE, a novel pipeline that enables simultan...

Semantic 3D-aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field

Recently 3D-aware GAN methods with neural radiance field have developed ...

Open Set Classification of GAN-based Image Manipulations via a ViT-based Hybrid Architecture

Classification of AI-manipulated content is receiving great attention, f...

Please sign up or login with your details

Forgot password? Click here to reset