Leveraging Pre-trained Language Model for Speech Sentiment Analysis

by   Suwon Shon, et al.

In this paper, we explore the use of pre-trained language models to learn sentiment information of written texts for speech sentiment analysis. First, we investigate how useful a pre-trained language model would be in a 2-step pipeline approach employing Automatic Speech Recognition (ASR) and transcripts-based sentiment analysis separately. Second, we propose a pseudo label-based semi-supervised training strategy using a language model on an end-to-end speech sentiment approach to take advantage of a large, but unlabeled speech dataset for training. Although spoken and written texts have different linguistic characteristics, they can complement each other in understanding sentiment. Therefore, the proposed system can not only model acoustic characteristics to bear sentiment-specific information in speech signals, but learn latent information to carry sentiments in the text representation. In these experiments, we demonstrate the proposed approaches improve F1 scores consistently compared to systems without a language model. Moreover, we also show that the proposed framework can reduce 65 supervision by leveraging a large amount of data without human sentiment annotation and boost performance in a low-resource condition where the human sentiment annotation is not available enough.


page 1

page 2

page 3

page 4


Speech Sentiment Analysis via Pre-trained Features from End-to-end ASR Models

In this paper, we propose to use pre-trained features from end-to-end AS...

WaBERT: A Low-resource End-to-end Model for Spoken Language Understanding and Speech-to-BERT Alignment

Historically lower-level tasks such as automatic speech recognition (ASR...

Keywords Extraction and Sentiment Analysis using Automatic Speech Recognition

Automatic Speech Recognition (ASR) is the interdisciplinary subfield of ...

Connecting Humanities and Social Sciences: Applying Language and Speech Technology to Online Panel Surveys

In this paper, we explore the application of language and speech technol...

EmotionGIF-Yankee: A Sentiment Classifier with Robust Model Based Ensemble Methods

This paper provides a method to classify sentiment with robust model bas...

Using LLM-assisted Annotation for Corpus Linguistics: A Case Study of Local Grammar Analysis

Chatbots based on Large Language Models (LLMs) have shown strong capabil...

Semi-supervised and Transfer learning approaches for low resource sentiment classification

Sentiment classification involves quantifying the affective reaction of ...

Please sign up or login with your details

Forgot password? Click here to reset