Improving Scholarly Knowledge Representation: Evaluating BERT-based Models for Scientific Relation Classification

by   Ming Jiang, et al.

With the rapid growth of research publications, there is a vast amount of scholarly knowledge that needs to be organized in digital libraries. To deal with this challenge, techniques relying on knowledge-graph structures are being advocated. Within such graph-based pipelines, inferring relation types between related scientific concepts is a crucial step. Recently, advanced techniques relying on language models pre-trained on the large corpus have been popularly explored for automatic relation classification. Despite remarkable contributions that have been made, many of these methods were evaluated under different scenarios, which limits their comparability. To this end, we present a thorough empirical evaluation on eight Bert-based classification models by focusing on two key factors: 1) Bert model variants, and 2) classification strategies. Experiments on three corpora show that domain-specific pre-training corpus benefits the Bert-based classification model to identify the type of scientific relations. Although the strategy of predicting a single relation each time achieves a higher classification accuracy than the strategy of identifying multiple relation types simultaneously in general, the latter strategy demonstrates a more consistent performance in the corpus with either a large or small size of annotations. Our study aims to offer recommendations to the stakeholders of digital libraries for selecting the appropriate technique to build knowledge-graph-based systems for enhanced scholarly information organization.


page 1

page 2

page 3

page 4


Evaluating BERT-based Scientific Relation Classifiers for Scholarly Knowledge Graph Construction on Digital Library Collections

The rapid growth of research publications has placed great demands on di...

Automated Mining of Leaderboards for Empirical AI Research

With the rapid growth of research publications, empowering scientists to...

Methodology for identifying study sites in scientific corpus

The TERRE-ISTEX project aims at identifying the evolution of research wo...

K-BERT: Enabling Language Representation with Knowledge Graph

Pre-trained language representation models, such as BERT, capture a gene...

LP-BERT: Multi-task Pre-training Knowledge Graph BERT for Link Prediction

Link prediction plays an significant role in knowledge graph, which is a...

Open Research Knowledge Graph: Towards Machine Actionability in Scholarly Communication

Despite improved digital access to scientific publications in the last d...

Incorporating Structured Sentences with Time-enhanced BERT for Fully-inductive Temporal Relation Prediction

Temporal relation prediction in incomplete temporal knowledge graphs (TK...

Please sign up or login with your details

Forgot password? Click here to reset