Solving A System Of Linear Equations By Randomized Orthogonal Projections

by   Alireza Entezari, et al.
University of Florida

Randomization has shown catalyzing effects in linear algebra with promising perspectives for tackling computational challenges in large-scale problems. For solving a system of linear equations, we demonstrate the convergence of a broad class of algorithms that at each step pick a subset of n equations at random and update the iterate with the orthogonal projection to the subspace those equations represent. We identify, in this context, a specific degree-n polynomial that non-linearly transforms the singular values of the system towards equalization. This transformation to singular values and the corresponding condition number then characterizes the expected convergence rate of iterations. As a consequence, our results specify the convergence rate of the stochastic gradient descent algorithm, in terms of the mini-batch size n, when used for solving systems of linear equations.


page 1

page 2

page 3

page 4


A Novel Stochastic Stratified Average Gradient Method: Convergence Rate and Its Complexity

SGD (Stochastic Gradient Descent) is a popular algorithm for large scale...

A Sharp Convergence Rate for the Asynchronous Stochastic Gradient Descent

We give a sharp convergence rate for the asynchronous stochastic gradien...

A Linearly-Convergent Stochastic L-BFGS Algorithm

We propose a new stochastic L-BFGS algorithm and prove a linear converge...

Randomized Orthogonal Projection Methods for Krylov Subspace Solvers

Randomized orthogonal projection methods (ROPMs) can be used to speed up...

Strong convergence of parabolic rate 1 of discretisations of stochastic Allen-Cahn-type equations

Consider the approximation of stochastic Allen-Cahn-type equations (i.e....

Distributed Linear Equations over Random Networks

Distributed linear algebraic equation over networks, where nodes hold a ...

Debiasing Stochastic Gradient Descent to handle missing values

A major caveat of large scale data is their incom-pleteness. We propose ...

Please sign up or login with your details

Forgot password? Click here to reset