Disentangle then Parse:Night-time Semantic Segmentation with Illumination Disentanglement

by   Zhixiang Wei, et al.

Most prior semantic segmentation methods have been developed for day-time scenes, while typically underperforming in night-time scenes due to insufficient and complicated lighting conditions. In this work, we tackle this challenge by proposing a novel night-time semantic segmentation paradigm, i.e., disentangle then parse (DTP). DTP explicitly disentangles night-time images into light-invariant reflectance and light-specific illumination components and then recognizes semantics based on their adaptive fusion. Concretely, the proposed DTP comprises two key components: 1) Instead of processing lighting-entangled features as in prior works, our Semantic-Oriented Disentanglement (SOD) framework enables the extraction of reflectance component without being impeded by lighting, allowing the network to consistently recognize the semantics under cover of varying and complicated lighting conditions. 2) Based on the observation that the illumination component can serve as a cue for some semantically confused regions, we further introduce an Illumination-Aware Parser (IAParser) to explicitly learn the correlation between semantics and lighting, and aggregate the illumination features to yield more precise predictions. Extensive experiments on the night-time segmentation task with various settings demonstrate that DTP significantly outperforms state-of-the-art methods. Furthermore, with negligible additional parameters, DTP can be directly used to benefit existing day-time methods for night-time segmentation.


page 1

page 3

page 6

page 7


Night-time Semantic Segmentation with a Large Real Dataset

Although huge progress has been made on semantic segmentation in recent ...

Illumination adaptive person reid based on teacher-student model and adversarial training

Most existing works in Person Re-identification (ReID) focus on settings...

GMLight: Lighting Estimation via Geometric Distribution Approximation

Lighting estimation from a single image is an essential yet challenging ...

Semantic Nighttime Image Segmentation with Synthetic Stylized Data, Gradual Adaptation and Uncertainty-Aware Evaluation

This work addresses the problem of semantic segmentation of nighttime im...

Adapting Semantic Segmentation Models for Changes in Illumination and Camera Perspective

Semantic segmentation using deep neural networks has been widely explore...

Deep Dynamic Cloud Lighting

Sky illumination is a core source of lighting in rendering, and a substa...

Neural Illumination: Lighting Prediction for Indoor Environments

This paper addresses the task of estimating the light arriving from all ...

Please sign up or login with your details

Forgot password? Click here to reset