DeepAI AI Chat
Log In Sign Up

Cross-view Transformers for real-time Map-view Semantic Segmentation

05/05/2022
by   Brady Zhou, et al.
0

We present cross-view transformers, an efficient attention-based model for map-view semantic segmentation from multiple cameras. Our architecture implicitly learns a mapping from individual camera views into a canonical map-view representation using a camera-aware cross-view attention mechanism. Each camera uses positional embeddings that depend on its intrinsic and extrinsic calibration. These embeddings allow a transformer to learn the mapping across different views without ever explicitly modeling it geometrically. The architecture consists of a convolutional image encoder for each view and cross-view transformer layers to infer a map-view semantic segmentation. Our model is simple, easily parallelizable, and runs in real-time. The presented architecture performs at state-of-the-art on the nuScenes dataset, with 4x faster inference speeds. Code is available at https://github.com/bradyz/cross_view_transformers.

READ FULL TEXT

page 1

page 3

page 7

page 8

06/27/2022

LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentation

Recent works in autonomous driving have widely adopted the bird's-eye-vi...
06/09/2019

Cross-view Semantic Segmentation for Sensing Surroundings

Sensing surroundings is ubiquitous and effortless to humans: It takes a ...
10/03/2021

Translating Images into Maps

We approach instantaneous mapping, converting images to a top-down view ...
06/09/2022

Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformer

Learning Bird's Eye View (BEV) representation from surrounding-view came...
04/15/2021

Geometry-Free View Synthesis: Transformers and no 3D Priors

Is a geometric model required to synthesize novel views from a single im...
03/21/2023

360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View

Seeing only a tiny part of the whole is not knowing the full circumstanc...
05/31/2021

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

We present SegFormer, a simple, efficient yet powerful semantic segmenta...