DeepAI AI Chat
Log In Sign Up

Genie: An Open Box Counterfactual Policy Estimator for Optimizing Sponsored Search Marketplace

by   Murat Ali Bayir, et al.

In this paper, we propose an offline counterfactual policy estimation framework called Genie to optimize Sponsored Search Marketplace. Genie employs an open box simulation engine with click calibration model to compute the KPI impact of any modification to the system. From the experimental results on Bing traffic, we showed that Genie performs better than existing observational approaches that employs randomized experiments for traffic slices that have frequent policy updates. We also show that Genie can be used to tune completely new policies efficiently without creating risky randomized experiments due to cold start problem. As time of today, Genie hosts more than 10000 optimization jobs yearly which runs more than 30 Million processing node hours of big data jobs for Bing Ads. For the last 3 years, Genie has been proven to be the one of the major platforms to optimize Bing Ads Marketplace due to its reliability under frequent policy changes and its efficiency to minimize risks in real experiments.


page 1

page 2

page 3

page 4


Scheduling to Optimize Sojourn Time of Successful Jobs

Deep neural networks training jobs and other iterative computations freq...

Variance-Optimal Augmentation Logging for Counterfactual Evaluation in Contextual Bandits

Methods for offline A/B testing and counterfactual learning are seeing r...

CAB: Continuous Adaptive Blending Estimator for Policy Evaluation and Learning

The ability to perform offline A/B-testing and off-policy learning using...

Unifying Online and Counterfactual Learning to Rank

Optimizing ranking systems based on user interactions is a well-studied ...

Differential Approximation and Sprinting for Multi-Priority Big Data Engines

Today's big data clusters based on the MapReduce paradigm are capable of...

Counterfactual Estimation and Optimization of Click Metrics for Search Engines

Optimizing an interactive system against a predefined online metric is p...