Learning a 3D Morphable Face Reflectance Model from Low-cost Data

by   Yuxuan Han, et al.

Modeling non-Lambertian effects such as facial specularity leads to a more realistic 3D Morphable Face Model. Existing works build parametric models for diffuse and specular albedo using Light Stage data. However, only diffuse and specular albedo cannot determine the full BRDF. In addition, the requirement of Light Stage data is hard to fulfill for the research communities. This paper proposes the first 3D morphable face reflectance model with spatially varying BRDF using only low-cost publicly-available data. We apply linear shiness weighting into parametric modeling to represent spatially varying specular intensity and shiness. Then an inverse rendering algorithm is developed to reconstruct the reflectance parameters from non-Light Stage data, which are used to train an initial morphable reflectance model. To enhance the model's generalization capability and expressive power, we further propose an update-by-reconstruction strategy to finetune it on an in-the-wild dataset. Experimental results show that our method obtains decent rendering results with plausible facial specularities. Our code is released \href{https://yxuhan.github.io/ReflectanceMM/index.html}{\textcolor{magenta}{here}}.


page 1

page 5

page 6

page 7

page 8

page 11

page 12

page 13


NeuFace: Realistic 3D Neural Face Rendering from Multi-view Images

Realistic face rendering from multi-view images is beneficial to various...

A Morphable Face Albedo Model

In this paper, we bring together two divergent strands of research: phot...

Landmark Detection and 3D Face Reconstruction for Caricature using a Nonlinear Parametric Model

Caricature is an artistic abstraction of the human face by distorting or...

AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"

Over the last years, with the advent of Generative Adversarial Networks ...

FNeVR: Neural Volume Rendering for Face Animation

Face animation, one of the hottest topics in computer vision, has achiev...

Light Stage Super-Resolution: Continuous High-Frequency Relighting

The light stage has been widely used in computer graphics for the past t...

Lightweight Real-time Makeup Try-on in Mobile Browsers with Tiny CNN Models for Facial Tracking

Recent works on convolutional neural networks (CNNs) for facial alignmen...

Code Repositories


Learning a 3D Morphable Face Reflectance Model from Low-cost Data, accepted by CVPR 2023.

view repo

Please sign up or login with your details

Forgot password? Click here to reset