Single Image Depth Prediction Made Better: A Multivariate Gaussian Take

03/31/2023
by   Ce Liu, et al.
0

Neural-network-based single image depth prediction (SIDP) is a challenging task where the goal is to predict the scene's per-pixel depth at test time. Since the problem, by definition, is ill-posed, the fundamental goal is to come up with an approach that can reliably model the scene depth from a set of training examples. In the pursuit of perfect depth estimation, most existing state-of-the-art learning techniques predict a single scalar depth value per-pixel. Yet, it is well-known that the trained model has accuracy limits and can predict imprecise depth. Therefore, an SIDP approach must be mindful of the expected depth variations in the model's prediction at test time. Accordingly, we introduce an approach that performs continuous modeling of per-pixel depth, where we can predict and reason about the per-pixel depth and its distribution. To this end, we model per-pixel scene depth using a multivariate Gaussian distribution. Moreover, contrary to the existing uncertainty modeling methods – in the same spirit, where per-pixel depth is assumed to be independent, we introduce per-pixel covariance modeling that encodes its depth dependency w.r.t all the scene points. Unfortunately, per-pixel depth covariance modeling leads to a computationally expensive continuous loss function, which we solve efficiently using the learned low-rank approximation of the overall covariance matrix. Notably, when tested on benchmark datasets such as KITTI, NYU, and SUN-RGB-D, the SIDP model obtained by optimizing our loss function shows state-of-the-art results. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.

READ FULL TEXT

page 1

page 3

page 6

page 14

page 15

page 16

page 17

02/13/2023

VA-DepthNet: A Variational Approach to Single Image Depth Prediction

We introduce VA-DepthNet, a simple, effective, and accurate deep neural ...
07/25/2019

MultiDepth: Single-Image Depth Estimation via Multi-Task Regression and Classification

We introduce MultiDepth, a novel training strategy and convolutional neu...
03/18/2019

Bilateral Cyclic Constraint and Adaptive Regularization for Unsupervised Monocular Depth Prediction

Supervised learning methods to infer (hypothesize) depth of a scene from...
01/19/2015

Coupled Depth Learning

In this paper we propose a method for estimating depth from a single ima...
12/12/2019

Inferring Distributions Over Depth from a Single Image

When building a geometric scene understanding system for autonomous vehi...
08/24/2020

DiverseNet: When One Right Answer is not Enough

Many structured prediction tasks in machine vision have a collection of ...
12/22/2022

Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography

Modern mobile burst photography pipelines capture and merge a short sequ...

Please sign up or login with your details

Forgot password? Click here to reset