IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation

by   Juze Zhang, et al.

This paper presents an inverse kinematic optimization layer (IKOL) for 3D human pose and shape estimation that leverages the strength of both optimization- and regression-based methods within an end-to-end framework. IKOL involves a nonconvex optimization that establishes an implicit mapping from an image's 3D keypoints and body shapes to the relative body-part rotations. The 3D keypoints and the body shapes are the inputs and the relative body-part rotations are the solutions. However, this procedure is implicit and hard to make differentiable. So, to overcome this issue, we designed a Gauss-Newton differentiation (GN-Diff) procedure to differentiate IKOL. GN-Diff iteratively linearizes the nonconvex objective function to obtain Gauss-Newton directions with closed form solutions. Then, an automatic differentiation procedure is directly applied to generate a Jacobian matrix for end-to-end training. Notably, the GN-Diff procedure works fast because it does not rely on a time-consuming implicit differentiation procedure. The twist rotation and shape parameters are learned from the neural networks and, as a result, IKOL has a much lower computational overhead than most existing optimization-based methods. Additionally, compared to existing regression-based methods, IKOL provides a more accurate mesh-image correspondence. This is because it iteratively reduces the distance between the keypoints and also enhances the reliability of the pose structures. Extensive experiments demonstrate the superiority of our proposed framework over a wide range of 3D human pose and shape estimation methods.


page 1

page 3

page 7


EllipBody: A Light-weight and Part-based Representation for Human Pose and Shape Recovery

Human pose and shape recovery is an important task in computer vision an...

HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation

Model-based 3D pose and shape estimation methods reconstruct a full 3D m...

HEMlets PoSh: Learning Part-Centric Heatmap Triplets for 3D Human Pose and Shape Estimation

Estimating 3D human pose from a single image is a challenging task. This...

DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare

We present DenseRaC, a novel end-to-end framework for jointly estimating...

imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Pose

We present imGHUM, the first holistic generative model of 3D human shape...

SMPLR: Deep SMPL reverse for 3D human pose and shape recovery

A recent trend in 3D human pose and shape estimation is to use deep lear...

Revitalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

We propose a novel sparse constrained formulation and from it derive a r...

Please sign up or login with your details

Forgot password? Click here to reset