Whatcha lookin' at? DeepLIFTing BERT's Attention in Question Answering

10/14/2019
by   Ekaterina Arkhangelskaia, et al.
0

There has been great success recently in tackling challenging NLP tasks by neural networks which have been pre-trained and fine-tuned on large amounts of task data. In this paper, we investigate one such model, BERT for question-answering, with the aim to analyze why it is able to achieve significantly better results than other models. We run DeepLIFT on the model predictions and test the outcomes to monitor shift in the attention values for input. We also cluster the results to analyze any possible patterns similar to human reasoning depending on the kind of input paragraph and question the model is trying to answer.

READ FULL TEXT

page 5

page 6

research
08/04/2019

Exploring Neural Net Augmentation to BERT for Question Answering on SQUAD 2.0

Enhancing machine capabilities to answer questions has been a topic of c...
research
10/19/2021

Ensemble ALBERT on SQuAD 2.0

Machine question answering is an essential yet challenging task in natur...
research
10/21/2022

LittleBird: Efficient Faster Longer Transformer for Question Answering

BERT has shown a lot of sucess in a wide variety of NLP tasks. But it ha...
research
02/19/2023

Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT

Recently, ChatGPT has attracted great attention, as it can generate flue...
research
12/04/2020

RPT: Relational Pre-trained Transformer Is Almost All You Need towards Democratizing Data Preparation

Can AI help automate human-easy but computer-hard data preparation tasks...
research
12/21/2022

Analyzing Semantic Faithfulness of Language Models via Input Intervention on Conversational Question Answering

Transformer-based language models have been shown to be highly effective...
research
03/18/2023

An Empirical Study of Pre-trained Language Models in Simple Knowledge Graph Question Answering

Large-scale pre-trained language models (PLMs) such as BERT have recentl...

Please sign up or login with your details

Forgot password? Click here to reset