Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans

by   Sida Peng, et al.

This paper addresses the challenge of novel view synthesis for a human performer from a very sparse set of camera views. Some recent works have shown that learning implicit neural representations of 3D scenes achieves remarkable view synthesis quality given dense input views. However, the representation learning will be ill-posed if the views are highly sparse. To solve this ill-posed problem, our key idea is to integrate observations over video frames. To this end, we propose Neural Body, a new human body representation which assumes that the learned neural representations at different frames share the same set of latent codes anchored to a deformable mesh, so that the observations across frames can be naturally integrated. The deformable mesh also provides geometric guidance for the network to learn 3D representations more efficiently. Experiments on a newly collected multi-view dataset show that our approach outperforms prior works by a large margin in terms of the view synthesis quality. We also demonstrate the capability of our approach to reconstruct a moving person from a monocular video on the People-Snapshot dataset. The code and dataset will be available at


page 1

page 2

page 3

page 4

page 5

page 7

page 9

page 10


Multi-view Human Body Mesh Translator

Existing methods for human mesh recovery mainly focus on single-view fra...

Pixel2ISDF: Implicit Signed Distance Fields based Human Body Model from Multi-view and Multi-pose Images

In this report, we focus on reconstructing clothed humans in the canonic...

NDF: Neural Deformable Fields for Dynamic Human Modelling

We propose Neural Deformable Fields (NDF), a new representation for dyna...

Representation learning of vertex heatmaps for 3D human mesh reconstruction from multi-view images

This study addresses the problem of 3D human mesh reconstruction from mu...

Dynamic View Synthesis from Dynamic Monocular Video

We present an algorithm for generating novel views at arbitrary viewpoin...

GRF: Learning a General Radiance Field for 3D Scene Representation and Rendering

We present a simple yet powerful implicit neural function that can repre...

Few-shot Neural Human Performance Rendering from Sparse RGBD Videos

Recent neural rendering approaches for human activities achieve remarkab...

Please sign up or login with your details

Forgot password? Click here to reset