Federated Composite Optimization

11/17/2020
by   Honglin Yuan, et al.
0

Federated Learning (FL) is a distributed learning paradigm which scales on-device learning collaboratively and privately. Standard FL algorithms such as Federated Averaging (FedAvg) are primarily geared towards smooth unconstrained settings. In this paper, we study the Federated Composite Optimization (FCO) problem, where the objective function in FL includes an additive (possibly) non-smooth component. Such optimization problems are fundamental to machine learning and arise naturally in the context of regularization (e.g., sparsity, low-rank, monotonicity, and constraint). To tackle this problem, we propose different primal/dual averaging approaches and study their communication and computation complexities. Of particular interest is Federated Dual Averaging (FedDualAvg), a federated variant of the dual averaging algorithm. FedDualAvg uses a novel double averaging procedure, which involves gradient averaging step in standard dual averaging and an average of client updates akin to standard federated averaging. Our theoretical analysis and empirical experiments demonstrate that FedDualAvg outperforms baselines for FCO.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/25/2023

Federated Composite Saddle Point Optimization

Federated learning (FL) approaches for saddle point problems (SPP) have ...
research
07/17/2022

Fast Composite Optimization and Statistical Recovery in Federated Learning

As a prevalent distributed learning paradigm, Federated Learning (FL) tr...
research
02/12/2020

Federated Clustering via Matrix Factorization Models: From Model Averaging to Gradient Sharing

Recently, federated learning (FL) has drawn significant attention due to...
research
10/31/2022

Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms

This paper focuses on Bayesian inference in a federated learning context...
research
06/22/2022

: Calibrating Global and Local Models via Federated Learning Beyond Consensus

In federated learning (FL), the objective of collaboratively learning a ...
research
09/04/2023

Composite federated learning with heterogeneous data

We propose a novel algorithm for solving the composite Federated Learnin...
research
06/20/2022

QuAFL: Federated Averaging Can Be Both Asynchronous and Communication-Efficient

Federated Learning (FL) is an emerging paradigm to enable the large-scal...

Please sign up or login with your details

Forgot password? Click here to reset