Combining Data-driven Supervision with Human-in-the-loop Feedback for Entity Resolution

by   Wenpeng Yin, et al.

The distribution gap between training datasets and data encountered in production is well acknowledged. Training datasets are often constructed over a fixed period of time and by carefully curating the data to be labeled. Thus, training datasets may not contain all possible variations of data that could be encountered in real-world production environments. Tasked with building an entity resolution system - a model that identifies and consolidates data points that represent the same person - our first model exhibited a clear training-production performance gap. In this case study, we discuss our human-in-the-loop enabled, data-centric solution to closing the training-production performance divergence. We conclude with takeaways that apply to data-centric learning at large.


page 1

page 2

page 3

page 4


Performance Bounds for Pairwise Entity Resolution

One significant challenge to scaling entity resolution algorithms to mas...

DataPerf: Benchmarks for Data-Centric AI Development

Machine learning (ML) research has generally focused on models, while th...

Developing a Hybrid Data-Driven, Mechanistic Virtual Flow Meter – a Case Study

Virtual flow meters, mathematical models predicting production flow rate...

Measuring Stereotypes using Entity-Centric Data

Stereotypes inform how we present ourselves and others, and in turn how ...

Entity Abstraction in Visual Model-Based Reinforcement Learning

This paper tests the hypothesis that modeling a scene in terms of entiti...

Please sign up or login with your details

Forgot password? Click here to reset