research
∙
08/22/2023
Collect, Measure, Repeat: Reliability Factors for Responsible AI Data Collection
The rapid entry of machine learning approaches in our daily activities a...
share
research
∙
06/27/2023
"Is a picture of a bird a bird": Policy recommendations for dealing with ambiguity in machine vision models
Many questions that we ask about the world do not have a single clear an...
research
∙
05/22/2023
Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
The generative AI revolution in recent years has been spurred by an expa...
research
∙
02/16/2023
Human-Centered Responsible Artificial Intelligence: Current Future Trends
In recent years, the CHI community has seen significant growth in resear...
research
∙
01/23/2023
The Reasonable Effectiveness of Diverse Evaluation Data
In this paper, we present findings from an semi-experimental exploration...
research
∙
07/20/2022
DataPerf: Benchmarks for Data-Centric AI Development
Machine learning (ML) research has generally focused on models, while th...
research
∙
11/19/2021
Data Excellence for AI: Why Should You Care
The efficacy of machine learning (ML) models depends on both algorithms ...
research
∙
06/11/2021
Cross-replication Reliability – An Empirical Approach to Interpreting Inter-rater Reliability
We present a new approach to interpreting IRR that is empirical and cont...
research
∙
05/01/2020
Eliciting User Preferences for Personalized Explanations for Video Summaries
Video summaries or highlights are a compelling alternative for exploring...
research
∙
11/05/2019
Metrology for AI: From Benchmarks to Instruments
In this paper we present the first steps towards hardening the science o...
research
∙
04/12/2019
A Crowdsourced Frame Disambiguation Corpus with Ambiguity
We present a resource for the task of FrameNet semantic frame disambigua...
research
∙
09/24/2018
Empirical Methodology for Crowdsourcing Ground Truth
The process of gathering ground truth data through human annotation is a...
research
∙
09/03/2018
Crowdsourcing Semantic Label Propagation in Relation Classification
Distant supervision is a popular method for performing relation extracti...
research
∙
08/18/2018
CrowdTruth 2.0: Quality Metrics for Crowdsourcing with Disagreement
Typically crowdsourcing-based approaches to gather annotated data use in...
research
∙
05/01/2018
Capturing Ambiguity in Crowdsourcing Frame Disambiguation
FrameNet is a computational linguistics resource composed of semantic fr...
research
∙
11/14/2017
False Positive and Cross-relation Signals in Distant Supervision Data
Distant supervision (DS) is a well-established method for relation extra...
research
∙
01/09/2017