DeepAI AI Chat
Log In Sign Up

A Corpus for Modeling Word Importance in Spoken Dialogue Transcripts

by   Sushant Kafle, et al.
Rochester Institute of Technology

Motivated by a project to create a system for people who are deaf or hard-of-hearing that would use automatic speech recognition (ASR) to produce real-time text captions of spoken English during in-person meetings with hearing individuals, we have augmented a transcript of the Switchboard conversational dialogue corpus with an overlay of word-importance annotations, with a numeric score for each word, to indicate its importance to the meaning of each dialogue turn. Further, we demonstrate the utility of this corpus by training an automatic word importance labeling model; our best performing model has an F-score of 0.60 in an ordinal 6-class word-importance classification task with an agreement (concordance correlation coefficient) of 0.839 with the human annotators (agreement score between annotators is 0.89). Finally, we discuss our intended future applications of this resource, particularly for the task of evaluating ASR performance, i.e. creating metrics that predict ASR-output caption text usability for DHH users better thanWord Error Rate (WER).


page 1

page 2

page 3

page 4


Modeling Acoustic-Prosodic Cues for Word Importance Prediction in Spoken Dialogues

Prosodic cues in conversational speech aid listeners in discerning a mes...

Speech Corpus of Ainu Folklore and End-to-end Speech Recognition for Ainu Language

Ainu is an unwritten language that has been spoken by Ainu people who ar...

MASRI-HEADSET: A Maltese Corpus for Speech Recognition

Maltese, the national language of Malta, is spoken by approximately 500,...

Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing

The accuracy of Automated Speech Recognition (ASR) technology has improv...

Adapting Text-based Dialogue State Tracker for Spoken Dialogues

Although there have been remarkable advances in dialogue systems through...

Modeling ASR Ambiguity for Dialogue State Tracking Using Word Confusion Networks

Spoken dialogue systems typically use a list of top-N ASR hypotheses for...