Off-Policy Evaluation of Probabilistic Identity Data in Lookalike Modeling

by   Randell Cotta, et al.

We evaluate the impact of probabilistically-constructed digital identity data collected from Sep. to Dec. 2017 (approx.), in the context of Lookalike-targeted campaigns. The backbone of this study is a large set of probabilistically-constructed "identities", represented as small bags of cookies and mobile ad identifiers with associated metadata, that are likely all owned by the same underlying user. The identity data allows to generate "identity-based", rather than "identifier-based", user models, giving a fuller picture of the interests of the users underlying the identifiers. We employ off-policy techniques to evaluate the potential of identity-powered lookalike models without incurring the risk of allowing untested models to direct large amounts of ad spend or the large cost of performing A/B tests. We add to historical work on off-policy evaluation by noting a significant type of "finite-sample bias" that occurs for studies combining modestly-sized datasets and evaluation metrics involving rare events (e.g., conversions). We illustrate this bias using a simulation study that later informs the handling of inverse propensity weights in our analyses on real data. We demonstrate significant lift in identity-powered lookalikes versus an identity-ignorant baseline: on average 70 identifiers having little data themselves, but that can be inferred to belong to users with substantial data to aggregate across identifiers. This implies that identity-powered user modeling is especially important in the context of identifiers having very short lifespans (i.e., frequently churned cookies). Our work motivates and informs the use of probabilistically-constructed identities in marketing. It also deepens the canon of examples in which off-policy learning has been employed to evaluate the complex systems of the internet economy.


Self-sovereign identity as a tool for digital democracy

The importance of digital identity as a foundation for digital public se...

Identity Management through a global Discovery System based on Decentralized Identities

Digital identities today continue to be a company resource instead of be...

reclaimID: Secure, Self-Sovereign Identities using Name Systems and Attribute-Based Encryption

In this paper we present reclaimID: An architecture that allows users to...

Camera-Incremental Object Re-Identification with Identity Knowledge Evolution

Object Re-identification (ReID) aims to retrieve the probe object from m...

Towards Inclusive Avatars: Disability Representation in Avatar Platforms

Digital avatars are an important part of identity representation, but th...

Building K-Anonymous User Cohorts with Consecutive Consistent Weighted Sampling (CCWS)

To retrieve personalized campaigns and creatives while protecting user p...

Please sign up or login with your details

Forgot password? Click here to reset