Depth-supervised NeRF: Fewer Views and Faster Training for Free

by   Kangle Deng, et al.

One common failure mode of Neural Radiance Field (NeRF) models is fitting incorrect geometries when given an insufficient number of input views. We propose DS-NeRF (Depth-supervised Neural Radiance Fields), a loss for learning neural radiance fields that takes advantage of readily-available depth supervision. Our key insight is that sparse depth supervision can be used to regularize the learned geometry, a crucial component for effectively rendering novel views using NeRF. We exploit the fact that current NeRF pipelines require images with known camera poses that are typically estimated by running structure-from-motion (SFM). Crucially, SFM also produces sparse 3D points that can be used as “free" depth supervision during training: we simply add a loss to ensure that depth rendered along rays that intersect these 3D points is close to the observed depth. We find that DS-NeRF can render more accurate images given fewer training views while training 2-6x faster. With only two training views on real-world images, DS-NeRF significantly outperforms NeRF as well as other sparse-view variants. We show that our loss is compatible with these NeRF models, demonstrating that depth is a cheap and easily digestible supervisory signal. Finally, we show that DS-NeRF supports other types of depth supervision such as scanned depth sensors and RGBD reconstruction outputs.


page 1

page 2

page 7

page 8

page 9

page 12


SimpleNeRF: Regularizing Sparse Input Neural Radiance Fields with Simpler Solutions

Neural Radiance Fields (NeRF) show impressive performance for the photor...

Dense Depth Priors for Neural Radiance Fields from Sparse Input Views

Neural radiance fields (NeRF) encode a scene into a neural representatio...

ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fields

Neural radiance fields (NeRF) have achieved impressive performances in v...

SCADE: NeRFs from Space Carving with Ambiguity-Aware Depth Estimates

Neural radiance fields (NeRFs) have enabled high fidelity 3D reconstruct...

LF-Net: Learning Local Features from Images

We present a novel deep architecture and a training strategy to learn a ...

SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis

Neural Radiance Field (NeRF) significantly degrades when only a limited ...

Efficient Unpaired Image Dehazing with Cyclic Perceptual-Depth Supervision

Image dehazing without paired haze-free images is of immense importance,...

Please sign up or login with your details

Forgot password? Click here to reset