Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

by   Fan Wang, et al.

Image Super-Resolution (SR) provides a promising technique to enhance the image quality of low-resolution optical sensors, facilitating better-performing target detection and autonomous navigation in a wide range of robotics applications. It is noted that the state-of-the-art SR methods are typically trained and tested using single-channel inputs, neglecting the fact that the cost of capturing high-resolution images in different spectral domains varies significantly. In this paper, we attempt to leverage complementary information from a low-cost channel (visible/depth) to boost image quality of an expensive channel (thermal) using fewer parameters. To this end, we first present an effective method to virtually generate pixel-wise aligned visible and thermal images based on real-time 3D reconstruction of multi-modal data captured at various viewpoints. Then, we design a feature-level multispectral fusion residual network model to perform high-accuracy SR of thermal images by adaptively integrating co-occurrence features presented in multispectral images. Experimental results demonstrate that this new approach can effectively alleviate the ill-posed inverse problem of image SR by taking into account complementary information from an additional low-cost channel, significantly outperforming state-of-the-art SR approaches in terms of both accuracy and efficiency.


page 1

page 2

page 3

page 4

page 5

page 6

page 7


How Real is Real: Evaluating the Robustness of Real-World Super Resolution

Image super-resolution (SR) is a field in computer vision that focuses o...

CoReFusion: Contrastive Regularized Fusion for Guided Thermal Super-Resolution

Thermal imaging has numerous advantages over regular visible-range imagi...

Thermal Image Super-Resolution Using Second-Order Channel Attention with Varying Receptive Fields

Thermal images model the long-infrared range of the electromagnetic spec...

Fusion of Deep and Non-Deep Methods for Fast Super-Resolution of Satellite Images

In the emerging commercial space industry there is a drastic increase in...

Multi-FAN: Multi-Spectral Mosaic Super-Resolution Via Multi-Scale Feature Aggregation Network

This paper introduces a novel method to super-resolve multi-spectral ima...

XCycles Backprojection Acoustic Super-Resolution

The computer vision community has paid much attention to the development...

Photothermal-SR-Net: A Customized Deep Unfolding Neural Network for Photothermal Super Resolution Imaging

This paper presents deep unfolding neural networks to handle inverse pro...

Please sign up or login with your details

Forgot password? Click here to reset