Pruning-based Topology Refinement of 3D Mesh using a 2D Alpha Mask

by   Gaëtan Landreau, et al.

Image-based 3D reconstruction has increasingly stunning results over the past few years with the latest improvements in computer vision and graphics. Geometry and topology are two fundamental concepts when dealing with 3D mesh structures. But the latest often remains a side issue in the 3D mesh-based reconstruction literature. Indeed, performing per-vertex elementary displacements over a 3D sphere mesh only impacts its geometry and leaves the topological structure unchanged and fixed. Whereas few attempts propose to update the geometry and the topology, all need to lean on costly 3D ground-truth to determine the faces/edges to prune. We present in this work a method that aims to refine the topology of any 3D mesh through a face-pruning strategy that extensively relies upon 2D alpha masks and camera pose information. Our solution leverages a differentiable renderer that renders each face as a 2D soft map. Its pixel intensity reflects the probability of being covered during the rendering process by such a face. Based on the 2D soft-masks available, our method is thus able to quickly highlight all the incorrectly rendered faces for a given viewpoint. Because our module is agnostic to the network that produces the 3D mesh, it can be easily plugged into any self-supervised image-based (either synthetic or natural) 3D reconstruction pipeline to get complex meshes with a non-spherical topology.


page 8

page 9


Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks

Reconstructing the 3D mesh of a general object from a single image is no...

Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction

Rendering is the process of generating 2D images from 3D assets, simulat...

Adaptive Mesh Refinement for Topology Optimization with Discrete Geometric Components

This work introduces an Adaptive Mesh Refinement (AMR) strategy for the ...

SieveNet: Selecting Point-Based Features for Mesh Networks

Meshes are widely used in 3D computer vision and graphics, but their irr...

An End-to-End Geometric Deficiency Elimination Algorithm for 3D Meshes

The 3D mesh is an important representation of geometric data. In the gen...

REALY: Rethinking the Evaluation of 3D Face Reconstruction

The evaluation of 3D face reconstruction results typically relies on a r...

Multi-view Image-based Hand Geometry Refinement using Differentiable Monte Carlo Ray Tracing

The amount and quality of datasets and tools available in the research f...

Please sign up or login with your details

Forgot password? Click here to reset