A Unified Analysis of Federated Learning with Arbitrary Client Participation

05/26/2022
by   Shiqiang Wang, et al.
0

Federated learning (FL) faces challenges of intermittent client availability and computation/communication efficiency. As a result, only a small subset of clients can participate in FL at a given time. It is important to understand how partial client participation affects convergence, but most existing works have either considered idealized participation patterns or obtained results with non-zero optimality error for generic patterns. In this paper, we provide a unified convergence analysis for FL with arbitrary client participation. We first introduce a generalized version of federated averaging (FedAvg) that amplifies parameter updates at an interval of multiple FL rounds. Then, we present a novel analysis that captures the effect of client participation in a single term. By analyzing this term, we obtain convergence upper bounds for a wide range of participation patterns, including both non-stochastic and stochastic cases, which match either the lower bound of stochastic gradient descent (SGD) or the state-of-the-art results in specific settings. We also discuss various insights, recommendations, and experimental results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/12/2022

On Federated Learning with Energy Harvesting Clients

Catering to the proliferation of Internet of Things devices and distribu...
research
02/13/2022

On the Convergence of Clustered Federated Learning

In a federated learning system, the clients, e.g. mobile devices and org...
research
12/16/2022

Federated Learning with Flexible Control

Federated learning (FL) enables distributed model training from local da...
research
12/28/2021

Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback

In federated learning (FL) problems, client sampling plays a key role in...
research
06/06/2023

A Lightweight Method for Tackling Unknown Participation Probabilities in Federated Averaging

In federated learning (FL), clients usually have diverse participation p...
research
10/03/2022

Taming Fat-Tailed ("Heavier-Tailed” with Potentially Infinite Variance) Noise in Federated Learning

A key assumption in most existing works on FL algorithms' convergence an...
research
08/20/2021

Accelerating Federated Learning with a Global Biased Optimiser

Federated Learning (FL) is a recent development in the field of machine ...

Please sign up or login with your details

Forgot password? Click here to reset