Multi-Category Mesh Reconstruction From Image Collections

by   Alessandro Simoni, et al.

Recently, learning frameworks have shown the capability of inferring the accurate shape, pose, and texture of an object from a single RGB image. However, current methods are trained on image collections of a single category in order to exploit specific priors, and they often make use of category-specific 3D templates. In this paper, we present an alternative approach that infers the textured mesh of objects combining a series of deformable 3D models and a set of instance-specific deformation, pose, and texture. Differently from previous works, our method is trained with images of multiple object categories using only foreground masks and rough camera poses as supervision. Without specific 3D templates, the framework learns category-level models which are deformed to recover the 3D shape of the depicted object. The instance-specific deformations are predicted independently for each vertex of the learned 3D mesh, enabling the dynamic subdivision of the mesh during the training process. Experiments show that the proposed framework can distinguish between different object categories and learn category-specific shape priors in an unsupervised manner. Predicted shapes are smooth and can leverage from multiple steps of subdivision during the training process, obtaining comparable or state-of-the-art results on two public datasets. Models and code are publicly released.


page 8

page 15

page 16

page 17

page 18

page 19


Learning Category-Specific Mesh Reconstruction from Image Collections

We present a learning framework for recovering the 3D shape, camera, and...

Localization and Mapping using Instance-specific Mesh Models

This paper focuses on building semantic maps, containing object poses an...

WrappingNet: Mesh Autoencoder via Deep Sphere Deformation

There have been recent efforts to learn more meaningful representations ...

FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modelling

We investigate the use of Neural Radiance Fields (NeRF) to learn high qu...

Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency

Approaches for single-view reconstruction typically rely on viewpoint an...

Unsupervised 3D Shape Learning from Image Collections in the Wild

We present a method to learn the 3D surface of objects directly from a c...

Meshlet Priors for 3D Mesh Reconstruction

Estimating a mesh from an unordered set of sparse, noisy 3D points is a ...

Please sign up or login with your details

Forgot password? Click here to reset