Experimentation Platforms Meet Reinforcement Learning: Bayesian Sequential Decision-Making for Continuous Monitoring

by   Runzhe Wan, et al.

With the growing needs of online A/B testing to support the innovation in industry, the opportunity cost of running an experiment becomes non-negligible. Therefore, there is an increasing demand for an efficient continuous monitoring service that allows early stopping when appropriate. Classic statistical methods focus on hypothesis testing and are mostly developed for traditional high-stake problems such as clinical trials, while experiments at online service companies typically have very different features and focuses. Motivated by the real needs, in this paper, we introduce a novel framework that we developed in Amazon to maximize customer experience and control opportunity cost. We formulate the problem as a Bayesian optimal sequential decision making problem that has a unified utility function. We discuss extensively practical design choices and considerations. We further introduce how to solve the optimal decision rule via Reinforcement Learning and scale the solution. We show the effectiveness of this novel approach compared with existing methods via a large-scale meta-analysis on experiments in Amazon.


page 1

page 2

page 3

page 4


Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology

The rise of internet-based services and products in the late 1990's brou...

Sequential Bayesian experimental designs via reinforcement learning

Bayesian experimental design (BED) has been used as a method for conduct...

Meta-Learning surrogate models for sequential decision making

Meta-learning methods leverage past experience to learn data-driven indu...

An Opportunistic Bandit Approach for User Interface Experimentation

Facing growing competition from online rivals, the retail industry is in...

Rapid and Scalable Bayesian AB Testing

AB testing aids business operators with their decision making, and is co...

Inverse Reinforcement Learning for Sequential Hypothesis Testing and Search

This paper considers a novel formulation of inverse reinforcement learni...

Optimal Testing in the Experiment-rich Regime

Motivated by the widespread adoption of large-scale A/B testing in indus...

Please sign up or login with your details

Forgot password? Click here to reset