Learning-based Spatial and Angular Information Separation for Light Field Compression

by   Jinglei Shi, et al.

Light fields are a type of image data that capture both spatial and angular scene information by recording light rays emitted by a scene from different orientations. In this context, spatial information is defined as features that remain static regardless of perspectives, while angular information refers to features that vary between viewpoints. We propose a novel neural network that, by design, can separate angular and spatial information of a light field. The network represents spatial information using spatial kernels shared among all Sub-Aperture Images (SAIs), and angular information using sets of angular kernels for each SAI. To further improve the representation capability of the network without increasing parameter number, we also introduce angular kernel allocation and kernel tensor decomposition mechanisms. Extensive experiments demonstrate the benefits of information separation: when applied to the compression task, our network outperforms other state-of-the-art methods by a large margin. And angular information can be easily transferred to other scenes for rendering dense views, showing the successful separation and the potential use case for the view synthesis task. We plan to release the code upon acceptance of the paper to encourage further research on this topic.


page 1

page 6

page 8

page 13


Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression

Light field is a type of image data that captures the 3D scene informati...

OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation of Road Scenes

Light field cameras can provide rich angular and spatial information to ...

Stratified Labeling for Surface Consistent Parallax Correction and Occlusion Completion

The light field faithfully records the spatial and angular configuration...

Deep Sparse Light Field Refocusing

Light field photography enables to record 4D images, containing angular ...

Spatial-Angular Attention Network for Light Field Reconstruction

Learning-based light field reconstruction methods demand in constructing...

Prediction and Sampling with Local Graph Transforms for Quasi-Lossless Light Field Compression

Graph-based transforms have been shown to be powerful tools in terms of ...

Two-pass Light Field Image Compression for Spatial Quality and Angular Consistency

The quality assessment of light field images presents new challenges to ...

Please sign up or login with your details

Forgot password? Click here to reset