Theoretical Study and Comparison of SPSA and RDSA Algorithms

by   Yiwen Chen, et al.
The Johns Hopkins University

Stochastic approximation (SA) algorithms are widely used in system optimization problems when only noisy measurements of the system are available. This paper studies two types of SA algorithms in a multivariate Kiefer-Wolfowitz setting: random-direction SA (RDSA) and simultaneous-perturbation SA (SPSA), and then describes the bias term, convergence, and asymptotic normality of RDSA algorithms. The gradient estimations in RDSA and SPSA have different forms and, consequently, use different types of random perturbations. This paper looks at various valid distributions for perturbations in RDSA and SPSA and then compares the two algorithms using mean-square errors computed from asymptotic distribution. From both a theoretical and numerical point of view, we find that SPSA generally outperforms RDSA.


page 1

page 2

page 3

page 4


Asymptotic study of stochastic adaptive algorithm in non-convex landscape

This paper studies some asymptotic properties of adaptive algorithms wid...

Generalized Simultaneous Perturbation Stochastic Approximation with Reduced Estimator Bias

We present in this paper a family of generalized simultaneous perturbati...

Random directions stochastic approximation with deterministic perturbations

We introduce deterministic perturbation schemes for the recently propose...

A Multistep Lyapunov Approach for Finite-Time Analysis of Biased Stochastic Approximation

Motivated by the widespread use of temporal-difference (TD-) and Q-learn...

Error bound analysis of the stochastic parareal algorithm

Stochastic parareal (SParareal) is a probabilistic variant of the popula...

Estimate Sequences for Variance-Reduced Stochastic Composite Optimization

In this paper, we propose a unified view of gradient-based algorithms fo...

Geometry-informed irreversible perturbations for accelerated convergence of Langevin dynamics

We introduce a novel geometry-informed irreversible perturbation that ac...

Please sign up or login with your details

Forgot password? Click here to reset