Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning

02/17/2022
by   Yuxuan Sun, et al.
4

Federated edge learning (FEEL) is a promising distributed machine learning (ML) framework to drive edge intelligence applications. However, due to the dynamic wireless environments and the resource limitations of edge devices, communication becomes a major bottleneck. In this work, we propose time-correlated sparsification with hybrid aggregation (TCS-H) for communication-efficient FEEL, which exploits jointly the power of model compression and over-the-air computation. By exploiting the temporal correlations among model parameters, we construct a global sparsification mask, which is identical across devices, and thus enables efficient model aggregation over-the-air. Each device further constructs a local sparse vector to explore its own important parameters, which are aggregated via digital communication with orthogonal multiple access. We further design device scheduling and power allocation algorithms for TCS-H. Experiment results show that, under limited communication resources, TCS-H can achieve significantly higher accuracy compared to the conventional top-K sparsification with orthogonal model aggregation, with both i.i.d. and non-i.i.d. data distributions.

READ FULL TEXT
research
05/31/2021

Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraints

Machine learning and wireless communication technologies are jointly fac...
research
05/17/2023

Unsourced Massive Access-Based Digital Over-the-Air Computation for Efficient Federated Edge Learning

Over-the-air computation (OAC) is a promising technique to achieve fast ...
research
11/10/2021

Transmission Power Control for Over-the-Air Federated Averaging at Network Edge

Over-the-air computation (AirComp) has emerged as a new analog power-dom...
research
12/31/2018

Federated Learning via Over-the-Air Computation

The stringent requirements for low-latency and privacy of the emerging h...
research
02/28/2020

Federated Over-the-Air Subspace Learning from Incomplete Data

Federated learning refers to a distributed learning scenario in which us...
research
05/08/2022

Over-the-Air Federated Multi-Task Learning via Model Sparsification and Turbo Compressed Sensing

To achieve communication-efficient federated multitask learning (FMTL), ...
research
12/28/2022

Accelerating Distributed Optimization via Over-the-Air Computing

Distributed optimization is ubiquitous in emerging applications, such as...

Please sign up or login with your details

Forgot password? Click here to reset