On the Online Frank-Wolfe Algorithms for Convex and Non-convex Optimizations

10/05/2015
by   Jean Lafond, et al.
0

In this paper, the online variants of the classical Frank-Wolfe algorithm are considered. We consider minimizing the regret with a stochastic cost. The online algorithms only require simple iterative updates and a non-adaptive step size rule, in contrast to the hybrid schemes commonly considered in the literature. Several new results are derived for convex and non-convex losses. With a strongly convex stochastic cost and when the optimal solution lies in the interior of the constraint set or the constraint set is a polytope, the regret bound and anytime optimality are shown to be O( ^3 T / T ) and O( ^2 T / T), respectively, where T is the number of rounds played. These results are based on an improved analysis on the stochastic Frank-Wolfe algorithms. Moreover, the online algorithms are shown to converge even when the loss is non-convex, i.e., the algorithms find a stationary point to the time-varying/stochastic loss at a rate of O(√(1/T)). Numerical experiments on realistic data sets are presented to support our theoretical claims.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/16/2020

Projection-free Online Learning over Strongly Convex Sets

To efficiently solve online problems with complicated constraints, proje...
research
09/22/2019

A generalization of regularized dual averaging and its dynamics

Excessive computational cost for learning large data and streaming data ...
research
02/11/2023

Improved Dynamic Regret for Online Frank-Wolfe

To deal with non-stationary online problems with complex constraints, we...
research
04/12/2022

An Adaptive Time Stepping Scheme for Rate-Independent Systems with Non-Convex Energy

We investigate a local incremental stationary scheme for the numerical s...
research
04/04/2020

Tracking Performance of Online Stochastic Learners

The utilization of online stochastic algorithms is popular in large-scal...
research
10/16/2019

Dynamic Local Regret for Non-convex Online Forecasting

We consider online forecasting problems for non-convex machine learning ...
research
05/22/2020

Online Non-convex Learning for River Pollution Source Identification

In this paper, novel gradient based online learning algorithms are devel...

Please sign up or login with your details

Forgot password? Click here to reset