Object Semantics Give Us the Depth We Need: Multi-task Approach to Aerial Depth Completion

by   Sara Hatami Gazani, et al.

Depth completion and object detection are two crucial tasks often used for aerial 3D mapping, path planning, and collision avoidance of Uncrewed Aerial Vehicles (UAVs). Common solutions include using measurements from a LiDAR sensor; however, the generated point cloud is often sparse and irregular and limits the system's capabilities in 3D rendering and safety-critical decision-making. To mitigate this challenge, information from other sensors on the UAV (viz., a camera used for object detection) is utilized to help the depth completion process generate denser 3D models. Performing both aerial depth completion and object detection tasks while fusing the data from the two sensors poses a challenge to resource efficiency. We address this challenge by proposing a novel approach to jointly execute the two tasks in a single pass. The proposed method is based on an encoder-focused multi-task learning model that exposes the two tasks to jointly learned features. We demonstrate how semantic expectations of the objects in the scene learned by the object detection pathway can boost the performance of the depth completion pathway while placing the missing depth values. Experimental results show that the proposed multi-task network outperforms its single-task counterpart, particularly when exposed to defective inputs.


page 2

page 5

page 6

page 7


Multi-Task Multi-Sensor Fusion for 3D Object Detection

In this paper we propose to exploit multiple related tasks for accurate ...

LiDAR-guided object search and detection in Subterranean Environments

Detecting objects of interest, such as human survivors, safety equipment...

LUAI Challenge 2021 on Learning to Understand Aerial Images

This report summarizes the results of Learning to Understand Aerial Imag...

Hyperspherical Embedding for Point Cloud Completion

Most real-world 3D measurements from depth sensors are incomplete, and t...

LiDAR Meta Depth Completion

Depth estimation is one of the essential tasks to be addressed when crea...

OGMN: Occlusion-guided Multi-task Network for Object Detection in UAV Images

Occlusion between objects is one of the overlooked challenges for object...

Trajectory Prediction Path Planning for an Object Intercepting UAV with a Mounted Depth Camera

A novel control software architecture using ROS C++ is introduced fo...

Please sign up or login with your details

Forgot password? Click here to reset