Removing Non-Stationary Knowledge From Pre-Trained Language Models for Entity-Level Sentiment Classification in Finance

by   Guijin Son, et al.

Extraction of sentiment signals from news text, stock message boards, and business reports, for stock movement prediction, has been a rising field of interest in finance. Building upon past literature, the most recent works attempt to better capture sentiment from sentences with complex syntactic structures by introducing aspect-level sentiment classification (ASC). Despite the growing interest, however, fine-grained sentiment analysis has not been fully explored in non-English literature due to the shortage of annotated finance-specific data. Accordingly, it is necessary for non-English languages to leverage datasets and pre-trained language models (PLM) of different domains, languages, and tasks to best their performance. To facilitate finance-specific ASC research in the Korean language, we build KorFinASC, a Korean aspect-level sentiment classification dataset for finance consisting of 12,613 human-annotated samples, and explore methods of intermediate transfer learning. Our experiments indicate that past research has been ignorant towards the potentially wrong knowledge of financial entities encoded during the training phase, which has overestimated the predictive power of PLMs. In our work, we use the term "non-stationary knowledge” to refer to information that was previously correct but is likely to change, and present "TGT-Masking”, a novel masking pattern to restrict PLMs from speculating knowledge of the kind. Finally, through a series of transfer learning with TGT-Masking applied we improve 22.63 KorFinASC.


page 1

page 2

page 3

page 4


Solving Aspect Category Sentiment Analysis as a Text Generation Task

Aspect category sentiment analysis has attracted increasing research att...

SEntFiN 1.0: Entity-Aware Sentiment Analysis for Financial News

Fine-grained financial sentiment analysis on news headlines is a challen...

Chinese Fine-Grained Financial Sentiment Analysis with Large Language Models

Entity-level fine-grained sentiment analysis in the financial domain is ...

A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges

As an important fine-grained sentiment analysis problem, aspect-based se...

KESA: A Knowledge Enhanced Approach For Sentiment Analysis

Though some recent works focus on injecting sentiment knowledge into pre...

Tweet Sentiment Extraction using Viterbi Algorithm with Transfer Learning

Tweet sentiment extraction extracts the most significant portion of the ...

Pitfalls of Static Language Modelling

Our world is open-ended, non-stationary and constantly evolving; thus wh...

Please sign up or login with your details

Forgot password? Click here to reset