Dynamic High Resolution Deformable Articulated Tracking

by   Aaron Walsman, et al.
University of Washington

The last several years have seen significant progress in using depth cameras for tracking articulated objects such as human bodies, hands, and robotic manipulators. Most approaches focus on tracking skeletal parameters of a fixed shape model, which makes them insufficient for applications that require accurate estimates of deformable object surfaces. To overcome this limitation, we present a 3D model-based tracking system for articulated deformable objects. Our system is able to track human body pose and high resolution surface contours in real time using a commodity depth sensor and GPU hardware. We implement this as a joint optimization over a skeleton to account for changes in pose, and over the vertices of a high resolution mesh to track the subject's shape. Through experimental results we show that we are able to capture dynamic sub-centimeter surface detail such as folds and wrinkles in clothing. We also show that this shape estimation aids kinematic pose estimation by providing a more accurate target to match against the point cloud. The end result is highly accurate spatiotemporal and semantic information which is well suited for physical human robot interaction as well as virtual and augmented reality systems.


page 1

page 3

page 7

page 8


HRPose: Real-Time High-Resolution 6D Pose Estimation Network Using Knowledge Distillation

Real-time 6D object pose estimation is essential for many real-world app...

Robust Shape Estimation for 3D Deformable Object Manipulation

Existing shape estimation methods for deformable object manipulation suf...

The Phong Surface: Efficient 3D Model Fitting using Lifted Optimization

Realtime perceptual and interaction capabilities in mixed reality requir...

Egocentric 6-DoF Tracking of Small Handheld Objects

Virtual and augmented reality technologies have seen significant growth ...

High-Resolution Augmentation for Automatic Template-Based Matching of Human Models

We propose a new approach for 3D shape matching of deformable human shap...

DeepFormableTag: End-to-end Generation and Recognition of Deformable Fiducial Markers

Fiducial markers have been broadly used to identify objects or embed mes...

TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement

We present a novel model for Tracking Any Point (TAP) that effectively t...

Please sign up or login with your details

Forgot password? Click here to reset