Hierarchical Kinematic Probability Distributions for 3D Human Shape and Pose Estimation from Images in the Wild

by   Akash Sengupta, et al.

This paper addresses the problem of 3D human body shape and pose estimation from an RGB image. This is often an ill-posed problem, since multiple plausible 3D bodies may match the visual evidence present in the input - particularly when the subject is occluded. Thus, it is desirable to estimate a distribution over 3D body shape and pose conditioned on the input image instead of a single 3D reconstruction. We train a deep neural network to estimate a hierarchical matrix-Fisher distribution over relative 3D joint rotation matrices (i.e. body pose), which exploits the human body's kinematic tree structure, as well as a Gaussian distribution over SMPL body shape parameters. To further ensure that the predicted shape and pose distributions match the visual evidence in the input image, we implement a differentiable rejection sampler to impose a reprojection loss between ground-truth 2D joint coordinates and samples from the predicted distributions, projected onto the image plane. We show that our method is competitive with the state-of-the-art in terms of 3D shape and pose metrics on the SSP-3D and 3DPW datasets, while also yielding a structured probability distribution over 3D body shape and pose, with which we can meaningfully quantify prediction uncertainty and sample multiple plausible 3D reconstructions to explain a given input image. Code is available at https://github.com/akashsengupta1997/HierarchicalProbabilistic3DHuman .


page 1

page 3

page 6

page 9

page 12

page 13

page 14


HuManiFlow: Ancestor-Conditioned Normalising Flows on SO(3) Manifolds for Human Pose and Shape Distribution Estimation

Monocular 3D human pose and shape estimation is an ill-posed problem sin...

Probabilistic Estimation of 3D Human Shape and Pose with a Semantic Local Parametric Model

This paper addresses the problem of 3D human body shape and pose estimat...

Probabilistic Regression with Huber Distributions

In this paper we describe a probabilistic method for estimating the posi...

Deep3DPose: Realtime Reconstruction of Arbitrarily Posed Human Bodies from Single RGB Images

We introduce an approach that accurately reconstructs 3D human poses and...

3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data

We consider the problem of obtaining dense 3D reconstructions of humans ...

Probabilistic orientation estimation with matrix Fisher distributions

This paper focuses on estimating probability distributions over the set ...

MEEV: Body Mesh Estimation On Egocentric Video

This technical report introduces our solution, MEEV, proposed to the Ego...

Please sign up or login with your details

Forgot password? Click here to reset