FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity

by   Guanghao Li, et al.

Federated Learning (FL) enables training a global model without sharing the decentralized raw data stored on multiple devices to protect data privacy. Due to the diverse capacity of the devices, FL frameworks struggle to tackle the problems of straggler effects and outdated models. In addition, the data heterogeneity incurs severe accuracy degradation of the global model in the FL training process. To address aforementioned issues, we propose a hierarchical synchronous FL framework, i.e., FedHiSyn. FedHiSyn first clusters all available devices into a small number of categories based on their computing capacity. After a certain interval of local training, the models trained in different categories are simultaneously uploaded to a central server. Within a single category, the devices communicate the local updated model weights to each other based on a ring topology. As the efficiency of training in the ring topology prefers devices with homogeneous resources, the classification based on the computing capacity mitigates the impact of straggler effects. Besides, the combination of the synchronous update of multiple categories and the device communication within a single category help address the data heterogeneity issue while achieving high accuracy. We evaluate the proposed framework based on MNIST, EMNIST, CIFAR10 and CIFAR100 datasets and diverse heterogeneous settings of devices. Experimental results show that FedHiSyn outperforms six baseline methods, e.g., FedAvg, SCAFFOLD, and FedAT, in terms of training accuracy and efficiency.


page 1

page 2

page 3

page 4


HADFL: Heterogeneity-aware Decentralized Federated Learning Framework

Federated learning (FL) supports training models on geographically distr...

CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework

Federated learning (FL) is an emerging distributed machine learning para...

Asynchronous Federated Learning on Heterogeneous Devices: A Survey

Federated learning (FL) is experiencing a fast booming with the wave of ...

FedProto: Federated Prototype Learning over Heterogeneous Devices

The heterogeneity across devices usually hinders the optimization conver...

FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems

Federated Learning (FL) is a novel distributed machine learning which al...

Efficient Device Scheduling with Multi-Job Federated Learning

Recent years have witnessed a large amount of decentralized data in mult...

FedMAX: Mitigating Activation Divergence for Accurate and Communication-Efficient Federated Learning

In this paper, we identify a new phenomenon called activation-divergence...

Please sign up or login with your details

Forgot password? Click here to reset