research
∙
07/26/2023
Function Value Learning: Adaptive Learning Rates Based on the Polyak Stepsize and Function Splitting in ERM
Here we develop variants of SGD (stochastic gradient descent) with an ad...
share
research
∙
05/27/2023
A Model-Based Method for Minimizing CVaR and Beyond
We develop a variant of the stochastic prox-linear method for minimizing...
research
∙
01/12/2023
A Stochastic Proximal Polyak Step Size
Recently, the stochastic Polyak step size (SPS) has emerged as a competi...
research
∙
10/04/2022
Linear Convergence of Natural Policy Gradient Methods with Log-Linear Policies
We consider infinite-horizon discounted Markov decision processes and st...
research
∙
07/17/2022
SP2: A Second Order Stochastic Polyak Method
Recently the "SP" (Stochastic Polyak step size) method has emerged as a ...
research
∙
02/24/2022
Cutting Some Slack for SGD with Adaptive Polyak Stepsizes
Tuning the step size of stochastic gradient descent is tedious and error...
research
∙
07/23/2021
A general sample complexity analysis of vanilla policy gradient
The policy gradient (PG) is one of the most popular methods for solving ...
research
∙
06/22/2021
Stochastic Polyak Stepsize with a Moving Target
We propose a new stochastic gradient method that uses recorded past loss...
research
∙
06/19/2021
SAN: Stochastic Average Newton Algorithm for Minimizing Finite Sums
We present a principled approach for designing stochastic Newton methods...
research
∙
10/02/2020
Variance-Reduced Methods for Machine Learning
Stochastic optimization lies at the heart of machine learning, and its c...
research
∙
07/29/2020
A new framework for the computation of Hessians
We investigate the computation of Hessian matrices via Automatic Differe...
research
∙
06/22/2020
Sketched Newton-Raphson
We propose a new globally convergent stochastic second order method. Our...
research
∙
06/20/2020
Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimization
We present a unified theorem for the convergence analysis of stochastic ...
research
∙
06/18/2020
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation
We provide several convergence theorems for SGD for two large classes of...
research
∙
06/14/2020
On the convergence of the Stochastic Heavy Ball Method
We provide a comprehensive analysis of the Stochastic Heavy Ball (SHB) m...
research
∙
06/01/2020
Factorial Powers for Stochastic Optimization
The convergence rates for convex and non-convex optimization methods dep...
research
∙
02/26/2020
Fast Linear Convergence of Randomized BFGS
Since the late 1950's when quasi-Newton methods first appeared, they hav...
research
∙
07/31/2019
Towards closing the gap between the theory and practice of SVRG
Among the very first variance reduced stochastic methods for solving the...
research
∙
01/31/2019
Optimal mini-batch and step sizes for SAGA
Recently it has been shown that the step sizes of a family of variance r...
research
∙
03/04/2018
Greedy stochastic algorithms for entropy-regularized optimal transport problems
Optimal transport (OT) distances are finding evermore applications in ma...
research
∙
02/12/2018
Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order Optimization
We present the first accelerated randomized algorithm for solving linear...
research
∙
10/20/2017