MASK-CNN-Transformer For Real-Time Multi-Label Weather Recognition

by   Shengchao Chen, et al.

Weather recognition is an essential support for many practical life applications, including traffic safety, environment, and meteorology. However, many existing related works cannot comprehensively describe weather conditions due to their complex co-occurrence dependencies. This paper proposes a novel multi-label weather recognition model considering these dependencies. The proposed model called MASK-Convolutional Neural Network-Transformer (MASK-CT) is based on the Transformer, the convolutional process, and the MASK mechanism. The model employs multiple convolutional layers to extract features from weather images and a Transformer encoder to calculate the probability of each weather condition based on the extracted features. To improve the generalization ability of MASK-CT, a MASK mechanism is used during the training phase. The effect of the MASK mechanism is explored and discussed. The Mask mechanism randomly withholds some information from one-pair training instances (one image and its corresponding label). There are two types of MASK methods. Specifically, MASK-I is designed and deployed on the image before feeding it into the weather feature extractor and MASK-II is applied to the image label. The Transformer encoder is then utilized on the randomly masked image features and labels. The experimental results from various real-world weather recognition datasets demonstrate that the proposed MASK-CT model outperforms state-of-the-art methods. Furthermore, the high-speed dynamic real-time weather recognition capability of the MASK-CT is evaluated.


page 4

page 11

page 15

page 16

page 17

page 20


A CNN-RNN Architecture for Multi-Label Weather Recognition

Weather Recognition plays an important role in our daily lives and many ...

Dense-TNT: Efficient Vehicle Type Classification Neural Network Using Satellite Imagery

Accurate vehicle type classification serves a significant role in the in...

Label Mask for Multi-Label Text Classification

One of the key problems in multi-label text classification is how to tak...

Graph Attention Transformer Network for Multi-Label Image Classification

Multi-label classification aims to recognize multiple objects or attribu...

Mask Attention Networks: Rethinking and Strengthen Transformer

Transformer is an attention-based neural network, which consists of two ...

Learning Content-enhanced Mask Transformer for Domain Generalized Urban-Scene Segmentation

Domain-generalized urban-scene semantic segmentation (USSS) aims to lear...

MRVM-NeRF: Mask-Based Pretraining for Neural Radiance Fields

Most Neural Radiance Fields (NeRFs) have poor generalization ability, li...

Please sign up or login with your details

Forgot password? Click here to reset