SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view Stereopsis

by   Mengqi Ji, et al.
Tsinghua University

Multi-view stereopsis (MVS) tries to recover the 3D model from 2D images. As the observations become sparser, the significant 3D information loss makes the MVS problem more challenging. Instead of only focusing on densely sampled conditions, we investigate sparse-MVS with large baseline angles since the sparser sensation is more practical and more cost-efficient. By investigating various observation sparsities, we show that the classical depth-fusion pipeline becomes powerless for the case with a larger baseline angle that worsens the photo-consistency check. As another line of the solution, we present SurfaceNet+, a volumetric method to handle the 'incompleteness' and the 'inaccuracy' problems induced by a very sparse MVS setup. Specifically, the former problem is handled by a novel volume-wise view selection approach. It owns superiority in selecting valid views while discarding invalid occluded views by considering the geometric prior. Furthermore, the latter problem is handled via a multi-scale strategy that consequently refines the recovered geometry around the region with the repeating pattern. The experiments demonstrate the tremendous performance gap between SurfaceNet+ and state-of-the-art methods in terms of precision and recall. Under the extreme sparse-MVS settings in two datasets, where existing methods can only return very few points, SurfaceNet+ still works as well as in the dense MVS setting. The benchmark and the implementation are publicly available at


page 1

page 3

page 5

page 8

page 9

page 11

page 12


VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusion

Recent volumetric 3D reconstruction methods can produce very accurate re...

A Benchmark and a Baseline for Robust Multi-view Depth Estimation

Recent deep learning approaches for multi-view depth estimation are empl...

Neural Pixel Composition: 3D-4D View Synthesis from Multi-Views

We present Neural Pixel Composition (NPC), a novel approach for continuo...

Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking

In this paper, we propose an efficient and effective dense hybrid recurr...

ConsistentNeRF: Enhancing Neural Radiance Fields with 3D Consistency for Sparse View Synthesis

Neural Radiance Fields (NeRF) has demonstrated remarkable 3D reconstruct...

Scale-Consistent Fusion: from Heterogeneous Local Sampling to Global Immersive Rendering

Image-based geometric modeling and novel view synthesis based on sparse,...

Bringing Generalization to Deep Multi-view Detection

Multi-view Detection (MVD) is highly effective for occlusion reasoning a...

Please sign up or login with your details

Forgot password? Click here to reset