I See-Through You: A Framework for Removing Foreground Occlusion in Both Sparse and Dense Light Field Images

by   Jiwan Hur, et al.

Light field (LF) camera captures rich information from a scene. Using the information, the LF de-occlusion (LF-DeOcc) task aims to reconstruct the occlusion-free center view image. Existing LF-DeOcc studies mainly focus on the sparsely sampled (sparse) LF images where most of the occluded regions are visible in other views due to the large disparity. In this paper, we expand LF-DeOcc in more challenging datasets, densely sampled (dense) LF images, which are taken by a micro-lens-based portable LF camera. Due to the small disparity ranges of dense LF images, most of the background regions are invisible in any view. To apply LF-DeOcc in both LF datasets, we propose a framework, ISTY, which is defined and divided into three roles: (1) extract LF features, (2) define the occlusion, and (3) inpaint occluded regions. By dividing the framework into three specialized components according to the roles, the development and analysis can be easier. Furthermore, an explainable intermediate representation, an occlusion mask, can be obtained in the proposed framework. The occlusion mask is useful for comprehensive analysis of the model and other applications by manipulating the mask. In experiments, qualitative and quantitative results show that the proposed framework outperforms state-of-the-art LF-DeOcc methods in both sparse and dense LF datasets.


page 14

page 15

page 16

page 17

page 18

page 19

page 21

page 22


Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views

The extraction of a clean background image by removing foreground occlus...

DeOccNet: Learning to See Through Foreground Occlusions in Light Fields

Background objects occluded in some views of a light field (LF) camera c...

Smart Dimming Sunglasses for Photophobia Using Spatial Light Modulator

We propose a smart dimming sunglasses system for individuals with photop...

Variational Disparity Estimation Framework for Plenoptic Image

This paper presents a computational framework for accurately estimating ...

Stratified Labeling for Surface Consistent Parallax Correction and Occlusion Completion

The light field faithfully records the spatial and angular configuration...

Convolutional Transformer for Autonomous Recognition and Grading of Tomatoes Under Various Lighting, Occlusion, and Ripeness Conditions

Harvesting fully ripe tomatoes with mobile robots presents significant c...

Multiscale Dynamic Graph Representation for Biometric Recognition with Occlusions

Occlusion is a common problem with biometric recognition in the wild. Th...

Please sign up or login with your details

Forgot password? Click here to reset