Multiple Randomization Designs

by   Patrick Bajari, et al.

In this study we introduce a new class of experimental designs. In a classical randomized controlled trial (RCT), or A/B test, a randomly selected subset of a population of units (e.g., individuals, plots of land, or experiences) is assigned to a treatment (treatment A), and the remainder of the population is assigned to the control treatment (treatment B). The difference in average outcome by treatment group is an estimate of the average effect of the treatment. However, motivating our study, the setting for modern experiments is often different, with the outcomes and treatment assignments indexed by multiple populations. For example, outcomes may be indexed by buyers and sellers, by content creators and subscribers, by drivers and riders, or by travelers and airlines and travel agents, with treatments potentially varying across these indices. Spillovers or interference can arise from interactions between units across populations. For example, sellers' behavior may depend on buyers' treatment assignment, or vice versa. This can invalidate the simple comparison of means as an estimator for the average effect of the treatment in classical RCTs. We propose new experiment designs for settings in which multiple populations interact. We show how these designs allow us to study questions about interference that cannot be answered by classical randomized experiments. Finally, we develop new statistical methods for analyzing these Multiple Randomization Designs.


Sampling-based randomized designs for causal inference under the potential outcomes framework

We establish the inferential properties of the mean-difference estimator...

Natural Experiments

The term natural experiment is used inconsistently. In one interpretatio...

Synthetic Controls for Experimental Design

This article studies experimental design in settings where the experimen...

Heterogeneous Treatment Effects for Networks, Panels, and other Outcome Matrices

We are interested in the distribution of treatment effects for an experi...

Optimality of Matched-Pair Designs in Randomized Controlled Trials

In randomized controlled trials (RCTs), treatment is often assigned by s...

Randomization for the direct effect of an infectious disease intervention in a clustered study population

Randomized trials of infectious disease interventions often focus on pop...

Population Interference in Panel Experiments

The phenomenon of population interference, where a treatment assigned to...

Please sign up or login with your details

Forgot password? Click here to reset