Attention-Free Keyword Spotting

10/14/2021

∙

by   Mashrur M. Morshed, et al.

∙

0

∙

share

Till now, attention-based models have been used with great success in the keyword spotting problem domain. However, in light of recent advances in deep learning, the question arises whether self-attention is truly irreplaceable for recognizing speech keywords. We thus explore the usage of gated MLPs – previously shown to be alternatives to transformers in vision tasks – for the keyword spotting task. We verify our approach on the Google Speech Commands V2-35 dataset and show that it is possible to obtain performance comparable to the state of the art without any apparent usage of self-attention.