Learning Content-enhanced Mask Transformer for Domain Generalized Urban-Scene Segmentation

by   Qi Bi, et al.
University of Amsterdam

Domain-generalized urban-scene semantic segmentation (USSS) aims to learn generalized semantic predictions across diverse urban-scene styles. Unlike domain gap challenges, USSS is unique in that the semantic categories are often similar in different urban scenes, while the styles can vary significantly due to changes in urban landscapes, weather conditions, lighting, and other factors. Existing approaches typically rely on convolutional neural networks (CNNs) to learn the content of urban scenes. In this paper, we propose a Content-enhanced Mask TransFormer (CMFormer) for domain-generalized USSS. The main idea is to enhance the focus of the fundamental component, the mask attention mechanism, in Transformer segmentation models on content information. To achieve this, we introduce a novel content-enhanced mask attention mechanism. It learns mask queries from both the image feature and its down-sampled counterpart, as lower-resolution image features usually contain more robust content information and are less sensitive to style variations. These features are fused into a Transformer decoder and integrated into a multi-resolution content-enhanced mask attention learning scheme. Extensive experiments conducted on various domain-generalized urban-scene segmentation datasets demonstrate that the proposed CMFormer significantly outperforms existing CNN-based methods for domain-generalized semantic segmentation, achieving improvements of up to 14.00\% in terms of mIoU (mean intersection over union). The source code for CMFormer will be made available at this \href{https://github.com/BiQiWHU/domain-generalized-urban-scene-segmentation}{repository}.


page 2

page 4

page 7

page 9

page 17

page 18


WildNet: Learning Domain Generalized Semantic Segmentation from the Wild

We present a new domain generalized semantic segmentation network named ...

Efficient Hybrid Transformer: Learning Global-local Context for Urban Sence Segmentation

Semantic segmentation of fine-resolution urban scene images plays a vita...

FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segmentation

The RGB-Thermal (RGB-T) information for semantic segmentation has been e...

Feature Pyramid Network with Multi-Head Attention for Semantic Segmentation of Fine-Resolution Remotely Sensed Images

Semantic segmentation from fine-resolution remotely sensed images is an ...

Cars Can't Fly up in the Sky: Improving Urban-Scene Segmentation via Height-driven Attention Networks

This paper exploits the intrinsic features of urban-scene images and pro...

MASK-CNN-Transformer For Real-Time Multi-Label Weather Recognition

Weather recognition is an essential support for many practical life appl...

Please sign up or login with your details

Forgot password? Click here to reset