FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems

by   Rui Song, et al.

Bird's eye view (BEV) perception is becoming increasingly important in the field of autonomous driving. It uses multi-view camera data to learn a transformer model that directly projects the perception of the road environment onto the BEV perspective. However, training a transformer model often requires a large amount of data, and as camera data for road traffic is often private, it is typically not shared. Federated learning offers a solution that enables clients to collaborate and train models without exchanging data. In this paper, we propose FedBEVT, a federated transformer learning approach for BEV perception. We address two common data heterogeneity issues in FedBEVT: (i) diverse sensor poses and (ii) varying sensor numbers in perception systems. We present federated learning with camera-attentive personalization (FedCaP) and adaptive multi-camera masking (AMCM) to enhance the performance in real-world scenarios. To evaluate our method in real-world settings, we create a dataset consisting of four typical federated use cases. Our findings suggest that FedBEVT outperforms the baseline approaches in all four use cases, demonstrating the potential of our approach for improving BEV perception in autonomous driving. We will make all codes and data publicly available.


page 1

page 5

page 7

page 12

page 13

page 14

page 16

page 17


Deep Federated Learning for Autonomous Driving

Autonomous driving is an active research topic in both academia and indu...

Federated Deep Learning Meets Autonomous Vehicle Perception: Design and Verification

Realizing human-like perception is a challenge in open driving scenarios...

RMMDet: Road-Side Multitype and Multigroup Sensor Detection System for Autonomous Driving

Autonomous driving has now made great strides thanks to artificial intel...

FLAIR: Federated Learning Annotated Image Repository

Cross-device federated learning is an emerging machine learning (ML) par...

GOF-TTE: Generative Online Federated Learning Framework for Travel Time Estimation

Estimating the travel time of a path is an essential topic for intellige...

Federated Multi-View Learning for Private Medical Data Integration and Analysis

Along with the rapid expansion of information technology and digitalizat...

Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformer

Learning Bird's Eye View (BEV) representation from surrounding-view came...

Please sign up or login with your details

Forgot password? Click here to reset