On Explaining Confounding Bias

by   Brit Youngmann, et al.

When analyzing large datasets, analysts are often interested in the explanations for surprising or unexpected results produced by their queries. In this work, we focus on aggregate SQL queries that expose correlations in the data. A major challenge that hinders the interpretation of such queries is confounding bias, which can lead to an unexpected correlation. We generate explanations in terms of a set of confounding variables that explain the unexpected correlation observed in a query. We propose to mine candidate confounding variables from external sources since, in many real-life scenarios, the explanations are not solely contained in the input data. We present an efficient algorithm that finds the optimal subset of attributes (mined from external sources and the input dataset) that explain the unexpected correlation. This algorithm is embodied in a system called MESA. We demonstrate experimentally over multiple real-life datasets and through a user study that our approach generates insightful explanations, outperforming existing methods that search for explanations only in the input data. We further demonstrate the robustness of our system to missing data and the ability of MESA to handle input datasets containing millions of tuples and an extensive search space of candidate confounding attributes.


Putting Things into Context: Rich Explanations for Query Answers using Join Graphs (extended version)

In many data analysis applications, there is a need to explain why a sur...

To not miss the forest for the trees – a holistic approach for explaining missing answers over nested data (extended version)

Query-based explanations for missing answers identify which operators of...

Counterfactual Generation Under Confounding

A machine learning model, under the influence of observed or unobserved ...

Adjustment Criteria for Recovering Causal Effects from Missing Data

Confounding bias, missing data, and selection bias are three common obst...

Explaining Documents' Relevance to Search Queries

We present GenEx, a generative model to explain search results to users ...

Explain3D: Explaining Disagreements in Disjoint Datasets

Data plays an important role in applications, analytic processes, and ma...

HypDB: Detect, Explain And Resolve Bias in OLAP

On line analytical processing (OLAP) is an essential element of decision...

Please sign up or login with your details

Forgot password? Click here to reset