UniFed: A Benchmark for Federated Learning Frameworks

by   Xiaoyuan Liu, et al.
National University of Singapore
Zhejiang University
Shanghai Jiao Tong University
University of Illinois at Urbana-Champaign
Peking University
berkeley college

Federated Learning (FL) has become a practical and popular paradigm in machine learning. However, currently, there is no systematic solution that covers diverse use cases. Practitioners often face the challenge of how to select a matching FL framework for their use case. In this work, we present UniFed, the first unified benchmark for standardized evaluation of the existing open-source FL frameworks. With 15 evaluation scenarios, we present both qualitative and quantitative evaluation results of nine existing popular open-sourced FL frameworks, from the perspectives of functionality, usability, and system performance. We also provide suggestions on framework selection based on the benchmark conclusions and point out future improvement directions.


page 1

page 2

page 3

page 4


An Empirical Study of Bugs in Open-Source Federated Learning Framework

Federated learning (FL), as a decentralized machine learning solution to...

Model-Agnostic Federated Learning

Since its debut in 2016, Federated Learning (FL) has been tied to the in...

Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectives

Traditional Federated Learning (FL) follows a server-domincated cooperat...

FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimization

Hyperparameter optimization (HPO) is crucial for machine learning algori...

SLPerf: a Unified Framework for Benchmarking Split Learning

Data privacy concerns has made centralized training of data, which is sc...

AMLB: an AutoML Benchmark

Comparing different AutoML frameworks is notoriously challenging and oft...

The Applicability of Federated Learning to Official Statistics

This work investigates the potential of Federated Learning (FL) for offi...

Please sign up or login with your details

Forgot password? Click here to reset