BiasAsker: Measuring the Bias in Conversational AI System

by   Yuxuan Wan, et al.

Powered by advanced Artificial Intelligence (AI) techniques, conversational AI systems, such as ChatGPT and digital assistants like Siri, have been widely deployed in daily life. However, such systems may still produce content containing biases and stereotypes, causing potential social problems. Due to the data-driven, black-box nature of modern AI techniques, comprehensively identifying and measuring biases in conversational systems remains a challenging task. Particularly, it is hard to generate inputs that can comprehensively trigger potential bias due to the lack of data containing both social groups as well as biased properties. In addition, modern conversational systems can produce diverse responses (e.g., chatting and explanation), which makes existing bias detection methods simply based on the sentiment and the toxicity hardly being adopted. In this paper, we propose BiasAsker, an automated framework to identify and measure social bias in conversational AI systems. To obtain social groups and biased properties, we construct a comprehensive social bias dataset, containing a total of 841 groups and 8,110 biased properties. Given the dataset, BiasAsker automatically generates questions and adopts a novel method based on existence measurement to identify two types of biases (i.e., absolute bias and related bias) in conversational systems. Extensive experiments on 8 commercial systems and 2 famous research models, such as ChatGPT and GPT-3, show that 32.83 by BiasAsker can trigger biased behaviors in these widely deployed conversational systems. All the code, data, and experimental results have been released to facilitate future research.


page 3

page 9


Bias in Conversational Search: The Double-Edged Sword of the Personalized Knowledge Graph

Conversational AI systems are being used in personal devices, providing ...

Biased Embeddings from Wild Data: Measuring, Understanding and Removing

Many modern Artificial Intelligence (AI) systems make use of data embedd...

RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models

Text representation models are prone to exhibit a range of societal bias...

Towards Designing a ChatGPT Conversational Companion for Elderly People

Loneliness and social isolation are serious and widespread problems amon...

Conversational Group Detection With Deep Convolutional Networks

Detection of interacting and conversational groups from images has appli...

Artificial mental phenomena: Psychophysics as a framework to detect perception biases in AI models

Detecting biases in artificial intelligence has become difficult because...

Don't Discard All the Biased Instances: Investigating a Core Assumption in Dataset Bias Mitigation Techniques

Existing techniques for mitigating dataset bias often leverage a biased ...

Please sign up or login with your details

Forgot password? Click here to reset