Term Expansion and FinBERT fine-tuning for Hypernym and Synonym Ranking of Financial Terms

by   Ankush Chopra, et al.

Hypernym and synonym matching are one of the mainstream Natural Language Processing (NLP) tasks. In this paper, we present systems that attempt to solve this problem. We designed these systems to participate in the FinSim-3, a shared task of FinNLP workshop at IJCAI-2021. The shared task is focused on solving this problem for the financial domain. We experimented with various transformer based pre-trained embeddings by fine-tuning these for either classification or phrase similarity tasks. We also augmented the provided dataset with abbreviations derived from prospectus provided by the organizers and definitions of the financial terms from DBpedia [Auer et al., 2007], Investopedia, and the Financial Industry Business Ontology (FIBO). Our best performing system uses both FinBERT [Araci, 2019] and data augmentation from the afore-mentioned sources. We observed that term expansion using data augmentation in conjunction with semantic similarity is beneficial for this task and could be useful for the other tasks that deal with short phrases. Our best performing model (Accuracy: 0.917, Rank: 1.156) was developed by fine-tuning SentenceBERT [Reimers et al., 2019] (with FinBERT at the backend) over an extended labelled set created using the hierarchy of labels present in FIBO.


page 1

page 2

page 3

page 4


Detecting ESG topics using domain-specific language models and data augmentation approaches

Despite recent advances in deep learning-based language modelling, many ...

Learning Semantic Text Similarity to rank Hypernyms of Financial Terms

Over the years, there has been a paradigm shift in how users access fina...

Noise Stability Regularization for Improving BERT Fine-tuning

Fine-tuning pre-trained language models such as BERT has become a common...

Data Processing and Annotation Schemes for FinCausal Shared Task

This document explains the annotation schemes used to label the data for...

Fine-tuning Tree-LSTM for phrase-level sentiment classification on a Polish dependency treebank. Submission to PolEval task 2

We describe a variant of Child-Sum Tree-LSTM deep neural network (Tai et...

Yseop at FinSim-3 Shared Task 2021: Specializing Financial Domain Learning with Phrase Representations

In this paper, we present our approaches for the FinSim-3 Shared Task 20...

DICoE@FinSim-3: Financial Hypernym Detection using Augmented Terms and Distance-based Features

We present the submission of team DICoE for FinSim-3, the 3rd Shared Tas...

Please sign up or login with your details

Forgot password? Click here to reset