Using Image Priors to Improve Scene Understanding

by   Brigit Schroeder, et al.
University of California Santa Cruz

Semantic segmentation algorithms that can robustly segment objects across multiple camera viewpoints are crucial for assuring navigation and safety in emerging applications such as autonomous driving. Existing algorithms treat each image in isolation, but autonomous vehicles often revisit the same locations or maintain information from the immediate past. We propose a simple yet effective method for leveraging these image priors to improve semantic segmentation of images from sequential driving datasets. We examine several methods to fuse these temporal scene priors, and introduce a prior fusion network that is able to learn how to transfer this information. The prior fusion model improves the accuracy over the non-prior baseline from 69.1 73.3 to models such as FCN-8, our prior method achieves the same accuracy with 5 times fewer parameters. We used a simple encoder decoder backbone, but this general prior fusion method could be applied to more complex semantic segmentation backbones. We also discuss how structured representations of scenes in the form of a scene graph could be leveraged as priors to further improve scene understanding.


page 1

page 4


Robust 3D Scene Segmentation through Hierarchical and Learnable Part-Fusion

3D semantic segmentation is a fundamental building block for several sce...

TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistency

Semantic segmentation is a classic computer vision problem dedicated to ...

Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

Recent advancements in the perception for autonomous driving are driven ...

Incorporating Luminance, Depth and Color Information by Fusion-based Networks for Semantic Segmentation

Semantic segmentation is paramount to accomplish many scene understandin...

Learning Shape Priors by Pairwise Comparison for Robust Semantic Segmentation

Semantic segmentation is important in medical image analysis. Inspired b...

Semantic Road Layout Understanding by Generative Adversarial Inpainting

Autonomous driving is becoming a reality, yet vehicles still need to rel...

Multi-stream CNN based Video Semantic Segmentation for Automated Driving

Majority of semantic segmentation algorithms operate on a single frame e...

Please sign up or login with your details

Forgot password? Click here to reset