Flare-Aware Cross-modal Enhancement Network for Multi-spectral Vehicle Re-identification

by   Aihua Zheng, et al.

Multi-spectral vehicle re-identification aims to address the challenge of identifying vehicles in complex lighting conditions by incorporating complementary visible and infrared information. However, in harsh environments, the discriminative cues in RGB and NIR modalities are often lost due to strong flares from vehicle lamps or sunlight, and existing multi-modal fusion methods are limited in their ability to recover these important cues. To address this problem, we propose a Flare-Aware Cross-modal Enhancement Network that adaptively restores flare-corrupted RGB and NIR features with guidance from the flare-immunized thermal infrared spectrum. First, to reduce the influence of locally degraded appearance due to intense flare, we propose a Mutual Flare Mask Prediction module to jointly obtain flare-corrupted masks in RGB and NIR modalities in a self-supervised manner. Second, to use the flare-immunized TI information to enhance the masked RGB and NIR, we propose a Flare-Aware Cross-modal Enhancement module that adaptively guides feature extraction of masked RGB and NIR spectra with prior flare-immunized knowledge from the TI spectrum. Third, to extract common informative semantic information from RGB and NIR, we propose an Inter-modality Consistency loss that enforces semantic consistency between the two modalities. Finally, to evaluate the proposed FACENet in handling intense flare, we introduce a new multi-spectral vehicle re-ID dataset, called WMVEID863, with additional challenges such as motion blur, significant background changes, and particularly intense flare degradation. Comprehensive experiments on both the newly collected dataset and public benchmark multi-spectral vehicle re-ID datasets demonstrate the superior performance of the proposed FACENet compared to state-of-the-art methods, especially in handling strong flares. The code and dataset will be released soon.


page 1

page 4

page 6

page 7


Multi-Modal Hybrid Learning and Sequential Training for RGB-T Saliency Detection

RGB-T saliency detection has emerged as an important computer vision tas...

Dynamic Enhancement Network for Partial Multi-modality Person Re-identification

Many existing multi-modality studies are based on the assumption of moda...

Multi-spectral Vehicle Re-identification with Cross-directional Consistency Network and a High-quality Benchmark

To tackle the challenge of vehicle re-identification (Re-ID) in complex ...

Edge-aware Guidance Fusion Network for RGB Thermal Scene Parsing

RGB thermal scene parsing has recently attracted increasing research int...

Interactive Context-Aware Network for RGB-T Salient Object Detection

Salient object detection (SOD) focuses on distinguishing the most conspi...

Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection

Integrating multispectral data in object detection, especially visible a...

Learning Modal-Invariant and Temporal-Memory for Video-based Visible-Infrared Person Re-Identification

Thanks for the cross-modal retrieval techniques, visible-infrared (RGB-I...

Please sign up or login with your details

Forgot password? Click here to reset