FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems

by   Li Li, et al.

Federated Learning (FL) is a novel distributed machine learning which allows thousands of edge devices to train model locally without uploading data concentrically to the server. But since real federated settings are resource-constrained, FL is encountered with systems heterogeneity which causes a lot of stragglers directly and then leads to significantly accuracy reduction indirectly. To solve the problems caused by systems heterogeneity, we introduce a novel self-adaptive federated framework FedSAE which adjusts the training task of devices automatically and selects participants actively to alleviate the performance degradation. In this work, we 1) propose FedSAE which leverages the complete information of devices' historical training tasks to predict the affordable training workloads for each device. In this way, FedSAE can estimate the reliability of each device and self-adaptively adjust the amount of training load per client in each round. 2) combine our framework with Active Learning to self-adaptively select participants. Then the framework accelerates the convergence of the global model. In our framework, the server evaluates devices' value of training based on their training loss. Then the server selects those clients with bigger value for the global model to reduce communication overhead. The experimental result indicates that in a highly heterogeneous system, FedSAE converges faster than FedAvg, the vanilla FL framework. Furthermore, FedSAE outperforms than FedAvg on several federated datasets - FedSAE improves test accuracy by 26.7 90.3


page 2

page 4

page 5

page 6

page 7

page 8

page 9

page 10


TiFL: A Tier-based Federated Learning System

Federated Learning (FL) enables learning a shared model across many clie...

Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning

Federated learning (FL) is a framework for machine learning across heter...

Sample Selection with Deadline Control for Efficient Federated Learning on Heterogeneous Clients

Federated Learning (FL) trains a machine learning model on distributed c...

Auxo: Heterogeneity-Mitigating Federated Learning via Scalable Client Clustering

Federated learning (FL) is an emerging machine learning (ML) paradigm th...

DynamicFL: Balancing Communication Dynamics and Client Manipulation for Federated Learning

Federated Learning (FL) is a distributed machine learning (ML) paradigm,...

FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity

Federated Learning (FL) enables training a global model without sharing ...

Memory-adaptive Depth-wise Heterogenous Federated Learning

Federated learning is a promising paradigm that allows multiple clients ...

Please sign up or login with your details

Forgot password? Click here to reset