Accelerated Alternating Minimization

by   Sergey Guminov, et al.

Alternating minimization (AM) optimization algorithms have been known for a long time and are of importance in machine learning problems, among which we are mostly motivated by approximating optimal transport distances. AM algorithms assume that the decision variable is divided into several blocks and minimization in each block can be done explicitly or cheaply with high accuracy. The ubiquitous Sinkhorn's algorithm can be seen as an alternating minimization algorithm for the dual to the entropy-regularized optimal transport problem. We introduce an accelerated alternating minimization method with a 1/k^2 convergence rate, where k is the iteration counter. This improves over known bound 1/k for general AM methods and for the Sinkhorn's algorithm. Moreover, our algorithm converges faster than gradient-type methods in practice as it is free of the choice of the step-size and is adaptive to the local smoothness of the problem. We show that the proposed method is primal-dual, meaning that if we apply it to a dual problem, we can reconstruct the solution of the primal problem with the same convergence rate. We apply our method to the entropy regularized optimal transport problem and show experimentally, that it outperforms Sinkhorn's algorithm.


page 1

page 2

page 3

page 4


Improved Rate of First Order Algorithms for Entropic Optimal Transport

This paper improves the state-of-the-art rate of a first-order algorithm...

On the Convergence of Projected Alternating Maximization for Equitable and Optimal Transport

This paper studies the equitable and optimal transport (EOT) problem, wh...

Accelerated Primal-Dual Coordinate Descent for Computational Optimal Transport

We propose and analyze a novel accelerated primal-dual coordinate descen...

Trust the Critics: Generatorless and Multipurpose WGANs with Initial Convergence Guarantees

Inspired by ideas from optimal transport theory we present Trust the Cri...

Regularized Step Directions in Conjugate Gradient Minimization for Machine Learning

Conjugate gradient minimization methods (CGM) and their accelerated vari...

A Fast and Accurate Splitting Method for Optimal Transport: Analysis and Implementation

We develop a fast and reliable method for solving large-scale optimal tr...

Consensus-based Distributed Discrete Optimal Transport for Decentralized Resource Matching

Optimal transport has been used extensively in resource matching to prom...

Please sign up or login with your details

Forgot password? Click here to reset