LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR

by   Florent Bartoccioni, et al.

Vision-based depth estimation is a key feature in autonomous systems, which often relies on a single camera or several independent ones. In such a monocular setup, dense depth is obtained with either additional input from one or several expensive LiDARs, e.g., with 64 beams, or camera-only methods, which suffer from scale-ambiguity and infinite-depth problems. In this paper, we propose a new alternative of densely estimating metric depth by combining a monocular camera with a light-weight LiDAR, e.g., with 4 beams, typical of today's automotive-grade mass-produced laser scanners. Inspired by recent self-supervised methods, we introduce a novel framework, called LiDARTouch, to estimate dense depth maps from monocular images with the help of “touches” of LiDAR, i.e., without the need for dense ground-truth depth. In our setup, the minimal LiDAR input contributes on three different levels: as an additional model's input, in a self-supervised LiDAR reconstruction objective function, and to estimate changes of pose (a key component of self-supervised depth estimation architectures). Our LiDARTouch framework achieves new state of the art in self-supervised depth estimation on the KITTI dataset, thus supporting our choices of integrating the very sparse LiDAR signal with other visual features. Moreover, we show that the use of a few-beam LiDAR alleviates scale ambiguity and infinite-depth issues that camera-only methods suffer from. We also demonstrate that methods from the fully-supervised depth-completion literature can be adapted to a self-supervised regime with a minimal LiDAR signal.


page 3

page 5

page 8

page 10

page 12

page 13


Monocular Differentiable Rendering for Self-Supervised 3D Object Detection

3D object detection from monocular images is an ill-posed problem due to...

DepthP+P: Metric Accurate Monocular Depth Estimation using Planar and Parallax

Current self-supervised monocular depth estimation methods are mostly ba...

Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications

In recent years, self-supervised methods for monocular depth estimation ...

SimuGAN: Unsupervised forward modeling and optimal design of a LIDAR Camera

Energy-saving LIDAR camera for short distances estimates an object's dis...

Calibrating Self-supervised Monocular Depth Estimation

In the recent years, many methods demonstrated the ability of neural net...

OCTraN: 3D Occupancy Convolutional Transformer Network in Unstructured Traffic Scenarios

Modern approaches for vision-centric environment perception for autonomo...

Robust Self-Supervised Extrinsic Self-Calibration

Autonomous vehicles and robots need to operate over a wide variety of sc...

Please sign up or login with your details

Forgot password? Click here to reset