3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning

by   Rumeysa Bodur, et al.

Manipulating facial expressions is a challenging task due to fine-grained shape changes produced by facial muscles and the lack of input-output pairs for supervised learning. Unlike previous methods using Generative Adversarial Networks (GAN), which rely on cycle-consistency loss or sparse geometry (landmarks) loss for expression synthesis, we propose a novel GAN framework to exploit 3D dense (depth and surface normals) information for expression manipulation. However, a large-scale dataset containing RGB images with expression annotations and their corresponding depth maps is not available. To this end, we propose to use an off-the-shelf state-of-the-art 3D reconstruction model to estimate the depth and create a large-scale RGB-Depth dataset after a manual data clean-up process. We utilise this dataset to minimise the novel depth consistency loss via adversarial learning (note we do not have ground truth depth maps for generated face images) and the depth categorical loss of synthetic data on the discriminator. In addition, to improve the generalisation and lower the bias of the depth parameters, we propose to use a novel confidence regulariser on the discriminator side of the framework. We extensively performed both quantitative and qualitative evaluations on two publicly available challenging facial expression benchmarks: AffectNet and RaFD. Our experiments demonstrate that the proposed method outperforms the competitive baseline and existing arts by a large margin.


page 1

page 4

page 5

page 8

page 12

page 13


Geometry Guided Adversarial Facial Expression Synthesis

Facial expression synthesis has drawn much attention in the field of com...

PortraitGAN for Flexible Portrait Manipulation

Previous methods have dealt with discrete manipulation of facial attribu...

Conditional Adversarial Synthesis of 3D Facial Action Units

Employing deep learning-based approaches for fine-grained facial express...

Explicit Facial Expression Transfer via Fine-Grained Semantic Representations

Facial expression transfer between two unpaired images is a challenging ...

3D Facial Geometry Recovery from a Depth View with Attention Guided Generative Adversarial Network

We present to recover the complete 3D facial geometry from a single dept...

Learning to Generate Facial Depth Maps

In this paper, an adversarial architecture for facial depth map estimati...

2D+3D Facial Expression Recognition via Discriminative Dynamic Range Enhancement and Multi-Scale Learning

In 2D+3D facial expression recognition (FER), existing methods generate ...

Please sign up or login with your details

Forgot password? Click here to reset