DeepAI AI Chat
Log In Sign Up

Single Image 3D Hand Reconstruction with Mesh Convolutions

by   Dominik Kulon, et al.

Monocular 3D reconstruction of deformable objects, such as human body parts, has been typically approached by predicting parameters of heavyweight linear models. In this paper, we demonstrate an alternative solution that is based on the idea of encoding images into a latent non-linear representation of meshes. The prior on 3D hand shapes is learned by training an autoencoder with intrinsic graph convolutions performed in the spectral domain. The pre-trained decoder acts as a non-linear statistical deformable model. The latent parameters that reconstruct the shape and articulated pose of hands in the image are predicted using an image encoder. We show that our system reconstructs plausible meshes and operates in real-time. We evaluate the quality of the mesh reconstructions produced by the decoder on a new dataset and show latent space interpolation results. Our code, data, and models will be made publicly available.


page 2

page 7

page 8

page 9


Monocular 3D Object Reconstruction with GAN Inversion

Recovering a textured 3D mesh from a monocular image is highly challengi...

Variational Autoencoders for Deforming 3D Mesh Models

3D geometric contents are becoming increasingly popular. In this paper, ...

Generating 3D faces using Convolutional Mesh Autoencoders

Learned 3D representations of human faces are useful for computer vision...

Learning 3D Human Body Embedding

Although human body shapes vary for different identities with different ...

Machine Learning to Predict Aerodynamic Stall

A convolutional autoencoder is trained using a database of airfoil aerod...

Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image

We develop a rotation equivariant model for generating 3D hand meshes fr...

WrappingNet: Mesh Autoencoder via Deep Sphere Deformation

There have been recent efforts to learn more meaningful representations ...

Code Repositories


Single Image 3D Hand Reconstruction with Mesh Convolutions

view repo