Monocular Real-time Full Body Capture with Inter-part Correlations

by   Yuxiao Zhou, et al.

We present the first method for real-time full body capture that estimates shape and motion of body and hands together with a dynamic 3D face model from a single color image. Our approach uses a new neural network architecture that exploits correlations between body and hands at high computational efficiency. Unlike previous works, our approach is jointly trained on multiple datasets focusing on hand, body or face separately, without requiring data where all the parts are annotated at the same time, which is much more difficult to create at sufficient variety. The possibility of such multi-dataset training enables superior generalization ability. In contrast to earlier monocular full body methods, our approach captures more expressive 3D face geometry and color by estimating the shape, expression, albedo and illumination parameters of a statistical face model. Our method achieves competitive accuracy on public benchmarks, while being significantly faster and providing more complete face reconstructions.


page 1

page 4

page 7


FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integration

Although the essential nuance of human motion is often conveyed as a com...

Expressive Body Capture: 3D Hands, Face, and Body from a Single Image

To facilitate the analysis of human actions, interactions and emotions, ...

Monocular Expressive Body Regression through Body-Driven Attention

To understand how people look, interact, or perform tasks, we need to qu...

Riggable 3D Face Reconstruction via In-Network Optimization

This paper presents a method for riggable 3D face reconstruction from mo...

XFormer: Fast and Accurate Monocular 3D Body Capture

We present XFormer, a novel human mesh and motion capture method that ac...

PERGAMO: Personalized 3D Garments from Monocular Video

Clothing plays a fundamental role in digital humans. Current approaches ...

YOLO-based Object Detection in Industry 4.0 Fischertechnik Model Environment

In this paper we extensively explore the suitability of YOLO architectur...

Please sign up or login with your details

Forgot password? Click here to reset