Deep Optical Coding Design in Computational Imaging

by   Henry Arguello, et al.

Computational optical imaging (COI) systems leverage optical coding elements (CE) in their setups to encode a high-dimensional scene in a single or multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Conventional approaches rely on random patterns or analytical designs to set the distribution of the CE. However, the available data and algorithm capabilities of deep neural networks (DNNs) have opened a new horizon in CE data-driven designs that jointly consider the optical encoder and computational decoder. Specifically, by modeling the COI measurements through a fully differentiable image formation model that considers the physics-based propagation of light and its interaction with the CEs, the parameters that define the CE and the computational decoder can be optimized in an end-to-end (E2E) manner. Moreover, by optimizing just CEs in the same framework, inference tasks can be performed from pure optics. This work surveys the recent advances on CE data-driven design and provides guidelines on how to parametrize different optical elements to include them in the E2E framework. Since the E2E framework can handle different inference applications by changing the loss function and the DNN, we present low-level tasks such as spectral imaging reconstruction or high-level tasks such as pose estimation with privacy preserving enhanced by using optimal task-based optical architectures. Finally, we illustrate classification and 3D object recognition applications performed at the speed of the light using all-optics DNN.


page 2

page 5

page 9

page 10

page 11

page 13

page 15

page 16


All-Optical Machine Learning Using Diffractive Deep Neural Networks

We introduce an all-optical Diffractive Deep Neural Network (D2NN) archi...

Computational Spectral Imaging: A Contemporary Overview

Spectral imaging collects and processes information along spatial and sp...

Neural 360^∘ Structured Light with Learned Metasurfaces

Structured light has proven instrumental in 3D imaging, LiDAR, and holog...

End-to-End Hyperspectral-Depth Imaging with Learned Diffractive Optics

To extend the capabilities of spectral imaging, hyperspectral and depth ...

A photosensor employing data-driven binning for ultrafast image recognition

Pixel binning is a technique, widely used in optical image acquisition a...

Diffractive optical system design by cascaded propagation

Modern design of complex optical systems relies heavily on computational...

Differentiable Microscopy Designs an All Optical Quantitative Phase Microscope

Ever since the first microscope by Zacharias Janssen in the late 16th ce...

Please sign up or login with your details

Forgot password? Click here to reset