Spectral Reconstruction and Disparity from Spatio-Spectrally Coded Light Fields via Multi-Task Deep Learning

by   Maximilian Schambach, et al.

We present a novel method to reconstruct a spectral central view and its aligned disparity map from spatio-spectrally coded light fields. Since we do not reconstruct an intermediate full light field from the coded measurement, we refer to this as principal reconstruction. The coded light fields correspond to those captured by a light field camera in the unfocused design with a spectrally coded microlens array. In this application, the spectrally coded light field camera can be interpreted as a single-shot spectral depth camera. We investigate several multi-task deep learning methods and propose a new auxiliary loss-based training strategy to enhance the reconstruction performance. The results are evaluated using a synthetic as well as a new real-world spectral light field dataset that we captured using a custom-built camera. The results are compared to state-of-the art compressed sensing reconstruction and disparity estimation. We achieve a high reconstruction quality for both synthetic and real-world coded light fields. The disparity estimation quality is on par with or even outperforms state-of-the-art disparity estimation from uncoded RGB light fields.


page 8

page 19

page 20

page 21

page 22

page 23

page 24

page 25


Learning Light Field Reconstruction from a Single Coded Image

Light field imaging is a rich way of representing the 3D world around us...

Fast Disparity Estimation from a Single Compressed Light Field Measurement

The abundant spatial and angular information from light fields has allow...

Non-Lambertian Surface Shape and Reflectance Reconstruction Using Concentric Multi-Spectral Light Field

Recovering the shape and reflectance of non-Lambertian surfaces remains ...

Regularizing disparity estimation via multi task learning with structured light reconstruction

3D reconstruction is a useful tool for surgical planning and guidance. H...

A Unified Learning Based Framework for Light Field Reconstruction from Coded Projections

Light field presents a rich way to represent the 3D world by capturing t...

VommaNet: an End-to-End Network for Disparity Estimation from Reflective and Texture-less Light Field Images

The precise combination of image sensor and micro-lens array enables len...

Hyperspectral Light Field Stereo Matching

In this paper, we describe how scene depth can be extracted using a hype...

Please sign up or login with your details

Forgot password? Click here to reset