Gradient Estimation with Discrete Stein Operators

by   Jiaxin Shi, et al.

Gradient estimation – approximating the gradient of an expectation with respect to the parameters of a distribution – is central to the solution of many machine learning problems. However, when the distribution is discrete, most common gradient estimators suffer from excessive variance. To improve the quality of gradient estimation, we introduce a variance reduction technique based on Stein operators for discrete distributions. We then use this technique to build flexible control variates for the REINFORCE leave-one-out estimator. Our control variates can be adapted online to minimize the variance and do not require extra evaluations of the target function. In benchmark generative modeling tasks such as training binary variational autoencoders, our gradient estimator achieves substantially lower variance than state-of-the-art estimators with the same number of function evaluations.


page 1

page 2

page 3

page 4


Double Control Variates for Gradient Estimation in Discrete Latent Variable Models

Stochastic gradient-based optimisation for discrete latent variable mode...

DBsurf: A Discrepancy Based Method for Discrete Stochastic Gradient Estimation

Computing gradients of an expectation with respect to the distributional...

VarGrad: A Low-Variance Gradient Estimator for Variational Inference

We analyse the properties of an unbiased gradient estimator of the ELBO ...

Generalized Doubly Reparameterized Gradient Estimators

Efficient low-variance gradient estimation enabled by the reparameteriza...

Training Discrete Deep Generative Models via Gapped Straight-Through Estimator

While deep generative models have succeeded in image processing, natural...

Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators

Discrete and especially binary random variables occur in many machine le...

Rao-Blackwellized Stochastic Gradients for Discrete Distributions

We wish to compute the gradient of an expectation over a finite or count...

Code Repositories


REINFORCE with Discrete Stein Operators

view repo

Please sign up or login with your details

Forgot password? Click here to reset