HeadNeRF: A Real-time NeRF-based Parametric Head Model

by   Yang Hong, et al.

In this paper, we propose HeadNeRF, a novel NeRF-based parametric head model that integrates the neural radiance field to the parametric representation of the human head. It can render high fidelity head images in real-time, and supports directly controlling the generated images' rendering pose and various semantic attributes. Different from existing related parametric models, we use the neural radiance fields as a novel 3D proxy instead of the traditional 3D textured mesh, which makes that HeadNeRF is able to generate high fidelity images. However, the computationally expensive rendering process of the original NeRF hinders the construction of the parametric NeRF model. To address this issue, we adopt the strategy of integrating 2D neural rendering to the rendering process of NeRF and design novel loss terms. As a result, the rendering speed of HeadNeRF can be significantly accelerated, and the rendering time of one frame is reduced from 5s to 25ms. The novel-designed loss terms also improve the rendering accuracy, and the fine-level details of the human head, such as the gaps between teeth, wrinkles, and beards, can be represented and synthesized by HeadNeRF. Extensive experimental results and several applications demonstrate its effectiveness. We will release the code and trained model to the public.


page 1

page 3

page 4

page 5

page 6

page 7

page 8


AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis

Generating high-fidelity talking head video by fitting with the input au...

Novel View Synthesis for High-fidelity Headshot Scenes

Rendering scenes with a high-quality human face from arbitrary viewpoint...

FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction

The advent of deep learning has led to significant progress in monocular...

FastNeRF: High-Fidelity Neural Rendering at 200FPS

Recent work on Neural Radiance Fields (NeRF) showed how neural networks ...

Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars

We propose a neural rendering-based system that creates head avatars fro...

Binaural Rendering of Ambisonic Signals by Neural Networks

Binaural rendering of ambisonic signals is of broad interest to virtual ...

NeRFocus: Neural Radiance Field for 3D Synthetic Defocus

Neural radiance fields (NeRF) bring a new wave for 3D interactive experi...

Please sign up or login with your details

Forgot password? Click here to reset