Uncalibrated Models Can Improve Human-AI Collaboration

by   Kailas Vodrahalli, et al.

In many practical applications of AI, an AI model is used as a decision aid for human users. The AI provides advice that a human (sometimes) incorporates into their decision-making process. The AI advice is often presented with some measure of "confidence" that the human can use to calibrate how much they depend on or trust the advice. In this paper, we demonstrate that presenting AI models as more confident than they actually are, even when the original AI is well-calibrated, can improve human-AI performance (measured as the accuracy and confidence of the human's final prediction after seeing the AI advice). We first learn a model for how humans incorporate AI advice using data from thousands of human interactions. This enables us to explicitly estimate how to transform the AI's prediction confidence, making the AI uncalibrated, in order to improve the final human prediction. We empirically validate our results across four different tasks – dealing with images, text and tabular data – involving hundreds of human participants. We further support our findings with simulation analysis. Our findings suggest the importance of and a framework for jointly optimizing the human-AI system as opposed to the standard paradigm of optimizing the AI model alone.


page 4

page 7


Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making

In AI-assisted decision-making, it is critical for human decision-makers...

Hierarchical Decision Ensembles- An inferential framework for uncertain Human-AI collaboration in forensic examinations

Forensic examination of evidence like firearms and toolmarks, traditiona...

Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making

Today, AI is being increasingly used to help human experts make decision...

Role of Human-AI Interaction in Selective Prediction

Recent work has shown the potential benefit of selective prediction syst...

Testing Human Ability To Detect Deepfake Images of Human Faces

Deepfakes are computationally-created entities that falsely represent re...

Using AI Uncertainty Quantification to Improve Human Decision-Making

AI Uncertainty Quantification (UQ) has the potential to improve human de...

Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare

The need for AI systems to provide explanations for their behaviour is n...

Please sign up or login with your details

Forgot password? Click here to reset