DeepAI AI Chat
Log In Sign Up

Finite-time High-probability Bounds for Polyak-Ruppert Averaged Iterates of Linear Stochastic Approximation

by   Alain Durmus, et al.

This paper provides a finite-time analysis of linear stochastic approximation (LSA) algorithms with fixed step size, a core method in statistics and machine learning. LSA is used to compute approximate solutions of a d-dimensional linear system 𝐀̅θ = 𝐛̅, for which (𝐀̅, 𝐛̅) can only be estimated through (asymptotically) unbiased observations {(𝐀(Z_n),𝐛(Z_n))}_n ∈ℕ. We consider here the case where {Z_n}_n ∈ℕ is an i.i.d. sequence or a uniformly geometrically ergodic Markov chain, and derive p-moments inequality and high probability bounds for the iterates defined by LSA and its Polyak-Ruppert averaged version. More precisely, we establish bounds of order (p α t_mix)^1/2d^1/p on the p-th moment of the last iterate of LSA. In this formula α is the step size of the procedure and t_mix is the mixing time of the underlying chain (t_mix=1 in the i.i.d. setting). We then prove finite-time instance-dependent bounds on the Polyak-Ruppert averaged sequence of iterates. These results are sharp in the sense that the leading term we obtain matches the local asymptotic minimax limit, including tight dependence on the parameters (d,t_mix) in the higher order terms.


page 1

page 2

page 3

page 4


Tight High Probability Bounds for Linear Stochastic Approximation with Fixed Stepsize

This paper provides a non-asymptotic analysis of linear stochastic appro...

Optimal and instance-dependent guarantees for Markovian linear stochastic approximation

We study stochastic approximation procedures for approximately solving a...

The ODE Method for Asymptotic Statistics in Stochastic Approximation and Reinforcement Learning

The paper concerns convergence and asymptotic statistics for stochastic ...

On TD(0) with function approximation: Concentration bounds and a centered variant with exponential convergence

We provide non-asymptotic bounds for the well-known temporal difference ...

Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noise

Linear two-timescale stochastic approximation (SA) scheme is an importan...

On Linear Stochastic Approximation: Fine-grained Polyak-Ruppert and Non-Asymptotic Concentration

We undertake a precise study of the asymptotic and non-asymptotic proper...