Using a single RGB frame for real time 3D hand pose estimation in the wild

by   Paschalis Panteleris, et al.

We present a method for the real-time estimation of the full 3D pose of one or more human hands using a single commodity RGB camera. Recent work in the area has displayed impressive progress using RGBD input. However, since the introduction of RGBD sensors, there has been little progress for the case of monocular color input. We capitalize on the latest advancements of deep learning, combining them with the power of generative hand pose estimation techniques to achieve real-time monocular 3D hand pose estimation in unrestricted scenarios. More specifically, given an RGB image and the relevant camera calibration information, we employ a state-of-the-art detector to localize hands. Given a crop of a hand in the image, we run the pretrained network of OpenPose for hands to estimate the 2D location of hand joints. Finally, non-linear least-squares minimization fits a 3D model of the hand to the estimated 2D joint positions, recovering the 3D hand pose. Extensive experimental results provide comparison to the state of the art as well as qualitative assessment of the method in the wild.


page 1

page 3

page 8


VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera

We present the first real-time method to capture the full global 3D skel...

Hand Pose Estimation via Latent 2.5D Heatmap Regression

Estimating the 3D pose of a hand is an essential part of human-computer ...

GraphPoseGAN: 3D Hand Pose Estimation from a Monocular RGB Image via Adversarial Learning on Graphs

This paper addresses the problem of 3D hand pose estimation from a monoc...

Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data

We present a novel method for monocular hand shape and pose estimation a...

EMG-Controlled Hand Teleoperation Using a Continuous Teleoperation Subspace

We present a method for EMG-driven teleoperation of non-anthropomorphic ...

Egocentric Pose Recognition in Four Lines of Code

We tackle the problem of estimating the 3D pose of an individual's upper...

Metric Pose Estimation for Human-Machine Interaction Using Monocular Vision

The rapid growth of collaborative robotics in production requires new au...

Please sign up or login with your details

Forgot password? Click here to reset