Evaluating OpenAI's Whisper ASR for Punctuation Prediction and Topic Modeling of life histories of the Museum of the Person

Automatic speech recognition (ASR) systems play a key role in applications involving human-machine interactions. Despite their importance, ASR models for the Portuguese language proposed in the last decade have limitations in relation to the correct identification of punctuation marks in automatic transcriptions, which hinder the use of transcriptions by other systems, models, and even by humans. However, recently Whisper ASR was proposed by OpenAI, a general-purpose speech recognition model that has generated great expectations in dealing with such limitations. This chapter presents the first study on the performance of Whisper for punctuation prediction in the Portuguese language. We present an experimental evaluation considering both theoretical aspects involving pausing points (comma) and complete ideas (exclamation, question, and fullstop), as well as practical aspects involving transcript-based topic modeling - an application dependent on punctuation marks for promising performance. We analyzed experimental results from videos of Museum of the Person, a virtual museum that aims to tell and preserve people's life histories, thus discussing the pros and cons of Whisper in a real-world scenario. Although our experiments indicate that Whisper achieves state-of-the-art results, we conclude that some punctuation marks require improvements, such as exclamation, semicolon and colon.


page 1

page 2

page 3

page 4


Challenges and Opportunities of Speech Recognition for Bengali Language

Speech recognition is a fascinating process that offers the opportunity ...

L2RS: A Learning-to-Rescore Mechanism for Automatic Speech Recognition

Modern Automatic Speech Recognition (ASR) systems primarily rely on scor...

Automatic Speech Recognition using limited vocabulary: A survey

Automatic Speech Recognition (ASR) is an active field of research due to...

Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator

Multi-talker overlapped speech poses a significant challenge for speech ...

Utterance-level neural confidence measure for end-to-end children speech recognition

Confidence measure is a performance index of particular importance for a...

Contextual ASR Adaptation for Conversational Agents

Statistical language models (LM) play a key role in Automatic Speech Rec...

A Survey Investigating Usage of Virtual Personal Assistants

Despite significant improvements in automatic speech recognition and spo...

Please sign up or login with your details

Forgot password? Click here to reset