On Testing of Samplers

by   Kuldeep S. Meel, et al.

Given a set of items ℱ and a weight function 𝚠𝚝: ℱ↦ (0,1), the problem of sampling seeks to sample an item proportional to its weight. Sampling is a fundamental problem in machine learning. The daunting computational complexity of sampling with formal guarantees leads designers to propose heuristics-based techniques for which no rigorous theoretical analysis exists to quantify the quality of generated distributions. This poses a challenge in designing a testing methodology to test whether a sampler under test generates samples according to a given distribution. Only recently, Chakraborty and Meel (2019) designed the first scalable verifier, called Barbarik1, for samplers in the special case when the weight function 𝚠𝚝 is constant, that is, when the sampler is supposed to sample uniformly from ℱ . The techniques in Barbarik1, however, fail to handle general weight functions. The primary contribution of this paper is an affirmative answer to the above challenge: motivated by Barbarik1 but using different techniques and analysis, we design Barbarik2 an algorithm to test whether the distribution generated by a sampler is ε-close or η-far from any target distribution. In contrast to black-box sampling techniques that require a number of samples proportional to |ℱ| , Barbarik2 requires only Õ(tilt(𝚠𝚝,φ)^2/η(η - 6ε)^3) samples, where the tilt is the maximum ratio of weights of two satisfying assignments. Barbarik2 can handle any arbitrary weight function. We present a prototype implementation of Barbarik2 and use it to test three state-of-the-art samplers.


page 1

page 2

page 3

page 4


INC: A Scalable Incremental Weighted Sampler

The fundamental problem of weighted sampling involves sampling of satisf...

Better Sum Estimation via Weighted Sampling

Given a large set U where each item a∈ U has weight w(a), we want to est...

WOR and p's: Sketches for ℓ_p-Sampling Without Replacement

Weighted sampling is a fundamental tool in data analysis and machine lea...

Distribution-Aware Sampling and Weighted Model Counting for SAT

Given a CNF formula and a weight for each assignment of values to variab...

Direct Sampling with a Step Function

The direct sampling method proposed by Walker et al. (JCGS 2011) can gen...

Stein Neural Sampler

We propose two novel samplers to produce high-quality samples from a giv...

Zonotope hit-and-run for efficient sampling from projection DPPs

Determinantal point processes (DPPs) are distributions over sets of item...

Please sign up or login with your details

Forgot password? Click here to reset