FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server

04/25/2022
by   Hong Zhang, et al.
Baidu, Inc.
Soochow University
0

Despite achieving remarkable performance, Federated Learning (FL) suffers from two critical challenges, i.e., limited computational resources and low training efficiency. In this paper, we propose a novel FL framework, i.e., FedDUAP, with two original contributions, to exploit the insensitive data on the server and the decentralized data in edge devices to further improve the training efficiency. First, a dynamic server update algorithm is designed to exploit the insensitive data on the server, in order to dynamically determine the optimal steps of the server update for improving the convergence and accuracy of the global model. Second, a layer-adaptive model pruning method is developed to perform unique pruning operations adapted to the different dimensions and importance of multiple layers, to achieve a good balance between efficiency and effectiveness. By integrating the two original techniques together, our proposed FL model, FedDUAP, significantly outperforms baseline approaches in terms of accuracy (up to 4.8 times faster), and computational cost (up to 61.9

READ FULL TEXT

page 1

page 2

page 3

page 4

09/18/2021

Toward Efficient Federated Learning in Multi-Channeled Mobile Edge Network with Layerd Gradient Compression

A fundamental issue for federated learning (FL) is how to achieve optima...
05/25/2022

Scalable and Low-Latency Federated Learning with Cooperative Mobile Edge Networking

Federated learning (FL) enables collaborative model training without cen...
01/27/2022

On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning

One of the biggest challenges in Federated Learning (FL) is that client ...
10/31/2019

Age-Based Scheduling Policy for Federated Learning in Mobile Edge Networks

Federated learning (FL) is a machine learning model that preserves data ...
07/18/2022

Federated Learning for Non-IID Data via Client Variance Reduction and Adaptive Server Update

Federated learning (FL) is an emerging technique used to collaboratively...
12/16/2021

DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systems

We study the problem of distributed training of neural networks (NNs) on...
11/08/2022

Federated Learning Using Three-Operator ADMM

Federated learning (FL) has emerged as an instance of distributed machin...

Please sign up or login with your details

Forgot password? Click here to reset