Binary Classification with Positive Labeling Sources

by   Jieyu Zhang, et al.
University of Washington

To create a large amount of training labels for machine learning models effectively and efficiently, researchers have turned to Weak Supervision (WS), which uses programmatic labeling sources rather than manual annotation. Existing works of WS for binary classification typically assume the presence of labeling sources that are able to assign both positive and negative labels to data in roughly balanced proportions. However, for many tasks of interest where there is a minority positive class, negative examples could be too diverse for developers to generate indicative labeling sources. Thus, in this work, we study the application of WS on binary classification tasks with positive labeling sources only. We propose WEAPO, a simple yet competitive WS method for producing training labels without negative labeling sources. On 10 benchmark datasets, we show WEAPO achieves the highest averaged performance in terms of both the quality of synthesized labels and the performance of the final classifier supervised with these labels. We incorporated the implementation of into WRENCH, an existing benchmarking platform.


page 1

page 2

page 3

page 4


Negative Confidence-Aware Weakly Supervised Binary Classification for Effective Review Helpfulness Classification

The incompleteness of positive labels and the presence of many unlabelle...

Binary classification with corrupted labels

In a binary classification problem where the goal is to fit an accurate ...

Adaptive Positive-Unlabelled Learning via Markov Diffusion

Positive-Unlabelled (PU) learning is the machine learning setting in whi...

Using novel data and ensemble models to improve automated labeling of Sustainable Development Goals

A number of labeling systems based on text have been proposed to help mo...

WRENCH: A Comprehensive Benchmark for Weak Supervision

Recent Weak Supervision (WS) approaches have had widespread success in e...

Binary Classification from Positive Data with Skewed Confidence

Positive-confidence (Pconf) classification [Ishida et al., 2018] is a pr...

Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels

Noisy PN learning is the problem of binary classification when training ...

Please sign up or login with your details

Forgot password? Click here to reset