FedDANE: A Federated Newton-Type Method

by   Tian Li, et al.
Carnegie Mellon University

Federated learning aims to jointly learn statistical models over massively distributed remote devices. In this work, we propose FedDANE, an optimization method that we adapt from DANE, a method for classical distributed optimization, to handle the practical constraints of federated learning. We provide convergence guarantees for this method when learning over both convex and non-convex functions. Despite encouraging theoretical results, we find that the method has underwhelming performance empirically. In particular, through empirical simulations on both synthetic and real-world datasets, FedDANE consistently underperforms baselines of FedAvg and FedProx in realistic federated settings. We identify low device participation and statistical device heterogeneity as two underlying causes of this underwhelming performance, and conclude by suggesting several directions of future work.


page 1

page 2

page 3

page 4


On the Convergence of Federated Optimization in Heterogeneous Networks

The burgeoning field of federated learning involves training machine lea...

One-Shot Federated Learning

We present one-shot federated learning, where a central server learns a ...

Second-Order Guarantees in Federated Learning

Federated learning is a useful framework for centralized learning from d...

Federated Learning: Challenges, Methods, and Future Directions

Federated learning involves training statistical models over remote devi...

Robust Networked Federated Learning for Localization

This paper addresses the problem of localization, which is inherently no...

Federated Distributionally Robust Optimization with Non-Convex Objectives: Algorithm and Analysis

Distributionally Robust Optimization (DRO), which aims to find an optima...

Robust Federated Learning: The Case of Affine Distribution Shifts

Federated learning is a distributed paradigm that aims at training model...

Please sign up or login with your details

Forgot password? Click here to reset