Deep Depth from Defocus: how can defocus blur improve 3D estimation using dense neural networks?

by   Marcela Carvalho, et al.

Depth estimation is of critical interest for scene understanding and accurate 3D reconstruction. Most recent approaches in depth estimation with deep learning exploit geometrical structures of standard sharp images to predict corresponding depth maps. However, cameras can also produce images with defocus blur depending on the depth of the objects and camera settings. Hence, these features may represent an important hint for learning to predict depth. In this paper, we propose a full system for single-image depth prediction in the wild using depth-from-defocus and neural networks. We carry out thorough experiments to test deep convolutional networks on real and simulated defocused images using a realistic model of blur variation with respect to depth. We also investigate the influence of blur on depth prediction observing model uncertainty with a Bayesian neural network approach. From these studies, we show that out-of-focus blur greatly improves the depth-prediction network performances. Furthermore, we transfer the ability learned on a synthetic, indoor dataset to real, indoor and outdoor images. For this purpose, we present a new dataset containing real all-focus and defocused images from a Digital Single-Lens Reflex (DSLR) camera, paired with ground truth depth maps obtained with an active 3D sensor for indoor scenes. The proposed approach is successfully validated on both this new dataset and standard ones as NYUv2 or Depth-in-the-Wild. Code and new datasets are available at


Blur aware metric depth estimation with multi-focus plenoptic cameras

While a traditional camera only captures one point of view of a scene, a...

Learning Depth from Focus in the Wild

For better photography, most recent commercial cameras including smartph...

Defocus Blur Detection via Depth Distillation

Defocus Blur Detection(DBD) aims to separate in-focus and out-of-focus r...

Learning to Dehaze From Realistic Scene with A Fast Physics Based Dehazing Network

Dehaze is one of the popular computer vision research topics for long. A...

Depth and DOF Cues Make A Better Defocus Blur Detector

Defocus blur detection (DBD) separates in-focus and out-of-focus regions...

Out-of-focus: Learning Depth from Image Bokeh for Robotic Perception

In this project, we propose a novel approach for estimating depth from R...

OmniHorizon: In-the-Wild Outdoors Depth and Normal Estimation from Synthetic Omnidirectional Dataset

Understanding the ambient scene is imperative for several applications s...

Code Repositories


Dense Deep Depth Estimation Network (D3-Net) in PyTorch.

view repo

Please sign up or login with your details

Forgot password? Click here to reset