Understanding and Mitigating the Effect of Outliers in Fair Ranking

by   Fatemeh Sarvi, et al.

Traditional ranking systems are expected to sort items in the order of their relevance and thereby maximize their utility. In fair ranking, utility is complemented with fairness as an optimization goal. Recent work on fair ranking focuses on developing algorithms to optimize for fairness, given position-based exposure. In contrast, we identify the potential of outliers in a ranking to influence exposure and thereby negatively impact fairness. An outlier in a list of items can alter the examination probabilities, which can lead to different distributions of attention, compared to position-based exposure. We formalize outlierness in a ranking, show that outliers are present in realistic datasets, and present the results of an eye-tracking study, showing that users scanning order and the exposure of items are influenced by the presence of outliers. We then introduce OMIT, a method for fair ranking in the presence of outliers. Given an outlier detection method, OMIT improves fair allocation of exposure by suppressing outliers in the top-k ranking. Using an academic search dataset, we show that outlierness optimization leads to a fairer policy that displays fewer outliers in the top-k, while maintaining a reasonable trade-off between fairness and utility.


Fairness of Exposure in Light of Incomplete Exposure Estimation

Fairness of exposure is a commonly used notion of fairness for ranking s...

Fair Ranking as Fair Division: Impact-Based Individual Fairness in Ranking

Rankings have become the primary interface in two-sided online markets. ...

Incentives for Item Duplication under Fair Ranking Policies

Ranking is a fundamental operation in information access systems, to fil...

On the Impact of Outlier Bias on User Clicks

User interaction data is an important source of supervision in counterfa...

Deep Clustering based Fair Outlier Detection

In this paper, we focus on the fairness issues regarding unsupervised ou...

Fair performance-based user recommendation in eCoaching systems

Offering timely support to users in eCoaching systems is a key factor to...

Reducing Disparate Exposure in Ranking: A Learning To Rank Approach

In this paper we consider a ranking problem in which we would like to or...

Please sign up or login with your details

Forgot password? Click here to reset