SemEval-2021 Task 11: NLPContributionGraph – Structuring Scholarly NLP Contributions for a Research Knowledge Graph

by   Jennifer D'Souza, et al.

There is currently a gap between the natural language expression of scholarly publications and their structured semantic content modeling to enable intelligent content search. With the volume of research growing exponentially every year, a search feature operating over semantically structured content is compelling. The SemEval-2021 Shared Task NLPContributionGraph (a.k.a. 'the NCG task') tasks participants to develop automated systems that structure contributions from NLP scholarly articles in the English language. Being the first-of-its-kind in the SemEval series, the task released structured data from NLP scholarly articles at three levels of information granularity, i.e. at sentence-level, phrase-level, and phrases organized as triples toward Knowledge Graph (KG) building. The sentence-level annotations comprised the few sentences about the article's contribution. The phrase-level annotations were scientific term and predicate phrases from the contribution sentences. Finally, the triples constituted the research overview KG. For the Shared Task, participating systems were then expected to automatically classify contribution sentences, extract scientific terms and relations from the sentences, and organize them as KG triples. Overall, the task drew a strong participation demographic of seven teams and 27 participants. The best end-to-end task system classified contribution sentences at 57.27 the absolute performance to generate triples remains low, in the conclusion of this article, the difficulty of producing such data and as a consequence of modeling it is highlighted.


page 1

page 2

page 3

page 4


Sentence, Phrase, and Triple Annotations to Build a Knowledge Graph of Natural Language Processing Contributions – A Trial Dataset

Purpose: The aim of this work is to normalize the NLPCONTRIBUTIONS schem...

UIUC_BioNLP at SemEval-2021 Task 11: A Cascade of Neural Models for Structuring Scholarly NLP Contributions

We propose a cascade of neural models that performs sentence classificat...

KnowGraph@IITK at SemEval-2021 Task 11: Building KnowledgeGraph for NLP Research

Research in Natural Language Processing is making rapid advances, result...

On the Composition of Scientific Abstracts

Scientific abstracts contain what is considered by the author(s) as info...

Knowledge Graph Extraction from Videos

Nearly all existing techniques for automated video annotation (or captio...

Research on multi-dimensional end-to-end phrase recognition algorithm based on background knowledge

At present, the deep end-to-end method based on supervised learning is u...

Contributions à l'asservissement visuel et à l'imagerie en médecine

This manuscript gives an overview of my research work carried out within...

Please sign up or login with your details

Forgot password? Click here to reset