Inexact and Stochastic Generalized Conditional Gradient with Augmented Lagrangian and Proximal Step

by   Antonio Silveti-Falls, et al.

In this paper we propose and analyze inexact and stochastic versions of the CGALP algorithm developed in the authors' previous paper, which we denote ICGALP, that allows for errors in the computation of several important quantities. In particular this allows one to compute some gradients, proximal terms, and/or linear minimization oracles in an inexact fashion that facilitates the practical application of the algorithm to computationally intensive settings, e.g. in high (or possibly infinite) dimensional Hilbert spaces commonly found in machine learning problems. The algorithm is able to solve composite minimization problems involving the sum of three convex proper lower-semicontinuous functions subject to an affine constraint of the form Ax=b for some bounded linear operator A. Only one of the functions in the objective is assumed to be differentiable, the other two are assumed to have an accessible prox operator and a linear minimization oracle. As main results, we show convergence of the Lagrangian to an optimum and asymptotic feasibility of the affine constraint as well as weak convergence of the dual variable to a solution of the dual problem, all in an almost sure sense. Almost sure convergence rates, both pointwise and ergodic, are given for the Lagrangian values and the feasibility gap. Numerical experiments verifying the predicted rates of convergence are shown as well.


page 1

page 2

page 3

page 4


Faster Projection-Free Augmented Lagrangian Methods via Weak Proximal Oracle

This paper considers a convex composite optimization problem with affine...

Global exponential stability of primal-dual gradient flow dynamics based on the proximal augmented Lagrangian: A Lyapunov-based approach

For a class of nonsmooth composite optimization problems with linear equ...

Super-Linear Convergence of Dual Augmented-Lagrangian Algorithm for Sparsity Regularized Estimation

We analyze the convergence behaviour of a recently proposed algorithm fo...

Stochastic Conditional Gradient Method for Composite Convex Minimization

In this paper, we propose the first practical algorithm to minimize stoc...

Frank-Wolfe Splitting via Augmented Lagrangian Method

Minimizing a function over an intersection of convex sets is an importan...

A Fully Stochastic Primal-Dual Algorithm

A new stochastic primal-dual algorithm for solving a composite optimizat...

Sub-linear convergence of a stochastic proximal iteration method in Hilbert space

We consider a stochastic version of the proximal point algorithm for opt...

Please sign up or login with your details

Forgot password? Click here to reset