Accelerating Transducers through Adjacent Token Merging

06/28/2023
by   Yuang Li, et al.
0

Recent end-to-end automatic speech recognition (ASR) systems often utilize a Transformer-based acoustic encoder that generates embedding at a high frame rate. However, this design is inefficient, particularly for long speech signals due to the quadratic computation of self-attention. To address this, we propose a new method, Adjacent Token Merging (A-ToMe), which gradually combines adjacent tokens with high similarity scores between their key values. In this way, the total time step could be reduced, and the inference of both the encoder and joint network is accelerated. Experiments on LibriSpeech show that our method can reduce 57 70 A-ToMe is also an effective solution to reduce tokens in long-form ASR, where the input speech consists of multiple utterances.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset