Pixel Adaptive Deep Unfolding Transformer for Hyperspectral Image Reconstruction

by   Miaoyu Li, et al.
Beijing Institute of Technology
Baidu, Inc.

Hyperspectral Image (HSI) reconstruction has made gratifying progress with the deep unfolding framework by formulating the problem into a data module and a prior module. Nevertheless, existing methods still face the problem of insufficient matching with HSI data. The issues lie in three aspects: 1) fixed gradient descent step in the data module while the degradation of HSI is agnostic in the pixel-level. 2) inadequate prior module for 3D HSI cube. 3) stage interaction ignoring the differences in features at different stages. To address these issues, in this work, we propose a Pixel Adaptive Deep Unfolding Transformer (PADUT) for HSI reconstruction. In the data module, a pixel adaptive descent step is employed to focus on pixel-level agnostic degradation. In the prior module, we introduce the Non-local Spectral Transformer (NST) to emphasize the 3D characteristics of HSI for recovering. Moreover, inspired by the diverse expression of features in different stages and depths, the stage interaction is improved by the Fast Fourier Transform (FFT). Experimental results on both simulated and real scenes exhibit the superior performance of our method compared to state-of-the-art HSI reconstruction methods. The code is released at: https://github.com/MyuLi/PADUT.


page 2

page 7

page 8


A Simple Adaptive Unfolding Network for Hyperspectral Image Reconstruction

We present a simple, efficient, and scalable unfolding network, SAUNet, ...

Local-Global Transformer Enhanced Unfolding Network for Pan-sharpening

Pan-sharpening aims to increase the spatial resolution of the low-resolu...

SST-ReversibleNet: Reversible-prior-based Spectral-Spatial Transformer for Efficient Hyperspectral Image Reconstruction

Spectral image reconstruction is an important task in snapshot compresse...

Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging

In coded aperture snapshot spectral compressive imaging (CASSI) systems,...

AdaptiveClick: Clicks-aware Transformer with Adaptive Focal Loss for Interactive Image Segmentation

Interactive Image Segmentation (IIS) has emerged as a promising techniqu...

PIP: Positional-encoding Image Prior

In Deep Image Prior (DIP), a Convolutional Neural Network (CNN) is fitte...

Multi-spectral Class Center Network for Face Manipulation Detection and Localization

As Deepfake contents continue to proliferate on the internet, advancing ...

Please sign up or login with your details

Forgot password? Click here to reset