Automatic Performance Estimation for Decentralized Optimization

03/11/2022
by   Sebastien Colla, et al.
0

We present a methodology to automatically compute worst-case performance bounds for a large class of first-order decentralized optimization algorithms. These algorithms aim at minimizing the average of local functions that are distributed across a network of agents. They typically combine local computations and consensus steps. Our methodology is based on the approach of Performance Estimation Problem (PEP), which allow computing the worst-case performance and worst-case instance of first-order optimization algorithms by solving an SDP. We propose two ways of representing consensus steps in PEPs, which allows writing and solving PEPs for decentralized optimization. The first formulation is exact but specific to a given averaging matrix. The second formulation is a relaxation that provides guarantees valid over an entire class of averaging matrices, characterized by their spectral range. This formulation often allows recovering the worst possible averaging matrix for the given algorithm. We apply our methodology to three different decentralized methods. For each of them, we obtain numerically tight worst-case performance bounds that significantly improve on the existing ones, as well as insights about the parameters tuning and the worst communication networks.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/26/2021

Automated Worst-Case Performance Analysis of Decentralized Gradient Descent

We develop a methodology to automatically compute worst-case performance...
10/03/2022

Automated Performance Estimation for Decentralized Optimization via Network Size Independent Problems

We develop a novel formulation of the Performance Estimation Problem (PE...
01/11/2022

PEPit: computer-assisted worst-case analyses of first-order optimization methods in Python

PEPit is a Python package aiming at simplifying the access to worst-case...
06/19/2020

Learning Optimal Power Flow: Worst-Case Guarantees for Neural Networks

This paper introduces for the first time a framework to obtain provable ...
11/03/2021

Average complexity of matrix reduction for clique filtrations

We study the algorithmic complexity of computing persistent homology of ...
12/10/2022

Stochastic Optimization for Spectral Risk Measures

Spectral risk objectives - also called L-risks - allow for learning syst...
02/03/2019

Stochastic first-order methods: non-asymptotic and computer-aided analyses via potential functions

We provide a novel computer-assisted technique for systematically analyz...

Please sign up or login with your details

Forgot password? Click here to reset