Individually Fair Learning with One-Sided Feedback

by   Yahav Bechavod, et al.

We consider an online learning problem with one-sided feedback, in which the learner is able to observe the true label only for positively predicted instances. On each round, k instances arrive and receive classification outcomes according to a randomized policy deployed by the learner, whose goal is to maximize accuracy while deploying individually fair policies. We first extend the framework of Bechavod et al. (2020), which relies on the existence of a human fairness auditor for detecting fairness violations, to instead incorporate feedback from dynamically-selected panels of multiple, possibly inconsistent, auditors. We then construct an efficient reduction from our problem of online learning with one-sided feedback and a panel reporting fairness violations to the contextual combinatorial semi-bandit problem (Cesa-Bianchi Lugosi, 2009, György et al., 2007). Finally, we show how to leverage the guarantees of two algorithms in the contextual combinatorial semi-bandit setting: Exp2 (Bubeck et al., 2012) and the oracle-efficient Context-Semi-Bandit-FTPL (Syrgkanis et al., 2016), to provide multi-criteria no regret guarantees simultaneously for accuracy and fairness. Our results eliminate two potential sources of bias from prior work: the "hidden outcomes" that are not available to an algorithm operating in the full information setting, and human biases that might be present in any single human auditor, but can be mitigated by selecting a well chosen panel.


page 1

page 2

page 3

page 4


Metric-Free Individual Fairness in Online Learning

We study an online learning problem subject to the constraint of individ...

Minimax Policies for Combinatorial Prediction Games

We address the online linear optimization problem when the actions of th...

Simple Combinatorial Algorithms for Combinatorial Bandits: Corruptions and Approximations

We consider the stochastic combinatorial semi-bandit problem with advers...

Cooperative Online Learning

In this preliminary (and unpolished) version of the paper, we study an a...

Online Learning with Set-Valued Feedback

We study a variant of online multiclass classification where the learner...

Efficient Learning with Arbitrary Covariate Shift

We give an efficient algorithm for learning a binary function in a given...

Equal Opportunity in Online Classification with Partial Feedback

We study an online classification problem with partial feedback in which...

Please sign up or login with your details

Forgot password? Click here to reset