NormalGAN: Learning Detailed 3D Human from a Single RGB-D Image

by   Lizhen Wang, et al.

We propose NormalGAN, a fast adversarial learning-based method to reconstruct the complete and detailed 3D human from a single RGB-D image. Given a single front-view RGB-D image, NormalGAN performs two steps: front-view RGB-D rectification and back-view RGBD inference. The final model was then generated by simply combining the front-view and back-view RGB-D information. However, inferring backview RGB-D image with high-quality geometric details and plausible texture is not trivial. Our key observation is: Normal maps generally encode much more information of 3D surface details than RGB and depth images. Therefore, learning geometric details from normal maps is superior than other representations. In NormalGAN, an adversarial learning framework conditioned by normal maps is introduced, which is used to not only improve the front-view depth denoising performance, but also infer the back-view depth image with surprisingly geometric details. Moreover, for texture recovery, we remove shading information from the front-view RGB image based on the refined normal map, which further improves the quality of the back-view color inference. Results and experiments on both testing data set and real captured data demonstrate the superior performance of our approach. Given a consumer RGB-D sensor, NormalGAN can generate the complete and detailed 3D human reconstruction results in 20 fps, which further enables convenient interactive experiences in telepresence, AR/VR and gaming scenarios.


page 1

page 2

page 3

page 4


PeelNet: Textured 3D reconstruction of human body using single view RGB image

Reconstructing human shape and pose from a single image is a challenging...

Deep Textured 3D Reconstruction of Human Bodies

Recovering textured 3D models of non-rigid human body shapes is challeng...

Monocular Human Digitization via Implicit Re-projection Networks

We present an approach to generating 3D human models from images. The ke...

BIPS: Bi-modal Indoor Panorama Synthesis via Residual Depth-aided Adversarial Learning

Providing omnidirectional depth along with RGB information is important ...

Deep Surface Normal Estimation with Hierarchical RGB-D Fusion

The growing availability of commodity RGB-D cameras has boosted the appl...

Ear-to-ear Capture of Facial Intrinsics

We present a practical approach to capturing ear-to-ear face models comp...

Normal Image Manipulation for Bas-relief Generation with Hybrid Styles

We introduce a normal-based bas-relief generation and stylization method...

Please sign up or login with your details

Forgot password? Click here to reset