Synthetic Training for Accurate 3D Human Pose and Shape Estimation in the Wild

by   Akash Sengupta, et al.

This paper addresses the problem of monocular 3D human shape and pose estimation from an RGB image. Despite great progress in this field in terms of pose prediction accuracy, state-of-the-art methods often predict inaccurate body shapes. We suggest that this is primarily due to the scarcity of in-the-wild training data with diverse and accurate body shape labels. Thus, we propose STRAPS (Synthetic Training for Real Accurate Pose and Shape), a system that utilises proxy representations, such as silhouettes and 2D joints, as inputs to a shape and pose regression neural network, which is trained with synthetic training data (generated on-the-fly during training using the SMPL statistical body model) to overcome data scarcity. We bridge the gap between synthetic training inputs and noisy real inputs, which are predicted by keypoint detection and segmentation CNNs at test-time, by using data augmentation and corruption during training. In order to evaluate our approach, we curate and provide a challenging evaluation dataset for monocular human shape estimation, Sports Shape and Pose 3D (SSP-3D). It consists of RGB images of tightly-clothed sports-persons with a variety of body shapes and corresponding pseudo-ground-truth SMPL shape and pose parameters, obtained via multi-frame optimisation. We show that STRAPS outperforms other state-of-the-art methods on SSP-3D in terms of shape prediction accuracy, while remaining competitive with the state-of-the-art on pose-centric datasets and metrics.


page 1

page 2

page 4

page 6

page 8

page 9

page 10


Probabilistic 3D Human Shape and Pose Estimation from Multiple Unconstrained Images in the Wild

This paper addresses the problem of 3D human body shape and pose estimat...

Accurate 3D Body Shape Regression using Metric and Semantic Attributes

While methods that regress 3D human meshes from images have progressed r...

DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare

We present DenseRaC, a novel end-to-end framework for jointly estimating...

Human Body Measurement Estimation with Adversarial Augmentation

We present a Body Measurement network (BMnet) for estimating 3D anthropo...

Deep Multitask Architecture for Integrated 2D and 3D Human Sensing

We propose a deep multitask architecture for fully automatic 2d and 3d h...

RGBD-Dog: Predicting Canine Pose from RGBD Sensors

The automatic extraction of animal 3D pose from images without markers i...

BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion

We show, for the first time, that neural networks trained only on synthe...

Please sign up or login with your details

Forgot password? Click here to reset