FedDec: Peer-to-peer Aided Federated Learning

by   Marina Costantini, et al.

Federated learning (FL) has enabled training machine learning models exploiting the data of multiple agents without compromising privacy. However, FL is known to be vulnerable to data heterogeneity, partial device participation, and infrequent communication with the server, which are nonetheless three distinctive characteristics of this framework. While much of the recent literature has tackled these weaknesses using different tools, only a few works have explored the possibility of exploiting inter-agent communication to improve FL's performance. In this work, we present FedDec, an algorithm that interleaves peer-to-peer communication and parameter averaging (similar to decentralized learning in networks) between the local gradient updates of FL. We analyze the convergence of FedDec under the assumptions of non-iid data distribution, partial device participation, and smooth and strongly convex costs, and show that inter-agent communication alleviates the negative impact of infrequent communication rounds with the server by reducing the dependence on the number of local updates H from O(H^2) to O(H). Furthermore, our analysis reveals that the term improved in the bound is multiplied by a constant that depends on the spectrum of the inter-agent communication graph, and that vanishes quickly the more connected the network is. We confirm the predictions of our theory in numerical simulations, where we show that FedDec converges faster than FedAvg, and that the gains are greater as either H or the connectivity of the network increase.


page 1

page 2

page 3

page 4


Hierarchical Federated Learning through LAN-WAN Orchestration

Federated learning (FL) was designed to enable mobile phones to collabor...

FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation

Federated Learning (FL) is a decentralized machine learning architecture...

Secure and Efficient Decentralized Federated Learning with Data Representation Protection

Federated learning (FL) is a promising technical support to the vision o...

Communication-Efficient Cluster Federated Learning in Large-scale Peer-to-Peer Networks

A traditional federated learning (FL) allows clients to collaboratively ...

A Newton-type algorithm for federated learning based on incremental Hessian eigenvector sharing

There is a growing interest in the decentralized optimization framework ...

Delay Sensitive Hierarchical Federated Learning with Stochastic Local Updates

The impact of local averaging on the performance of federated learning (...

FedExP: Speeding up Federated Averaging Via Extrapolation

Federated Averaging (FedAvg) remains the most popular algorithm for Fede...

Please sign up or login with your details

Forgot password? Click here to reset