Rethinking Document-Level Relation Extraction: A Reality Check

by   Jing Li, et al.
Harbin Institute of Technology

Recently, numerous efforts have continued to push up performance boundaries of document-level relation extraction (DocRE) and have claimed significant progress in DocRE. In this paper, we do not aim at proposing a novel model for DocRE. Instead, we take a closer look at the field to see if these performance gains are actually true. By taking a comprehensive literature review and a thorough examination of popular DocRE datasets, we find that these performance gains are achieved upon a strong or even untenable assumption in common: all named entities are perfectly localized, normalized, and typed in advance. Next, we construct four types of entity mention attacks to examine the robustness of typical DocRE models by behavioral probing. We also have a close check on model usability in a more realistic setting. Our findings reveal that most of current DocRE models are vulnerable to entity mention attacks and difficult to be deployed in real-world end-user NLP applications. Our study calls more attentions for future research to stop simplifying problem setups, and to model DocRE in the wild rather than in an unrealistic Utopian world.


A Masked Image Reconstruction Network for Document-level Relation Extraction

Document-level relation extraction aims to extract relations among entit...

Relation-Specific Attentions over Entity Mentions for Enhanced Document-Level Relation Extraction

Compared with traditional sentence-level relation extraction, document-l...

An Improved Baseline for Sentence-level Relation Extraction

Sentence-level relation extraction (RE) aims at identifying the relation...

Modeling Task Interactions in Document-Level Joint Entity and Relation Extraction

We target on the document-level relation extraction in an end-to-end set...

Improving Long Tailed Document-Level Relation Extraction via Easy Relation Augmentation and Contrastive Learning

Towards real-world information extraction scenario, research of relation...

Incorporating Consistency Verification into Neural Data-to-Document Generation

Recent neural models for data-to-document generation have achieved remar...

Please sign up or login with your details

Forgot password? Click here to reset