Improving Matching Models with Contextualized Word Representations for Multi-turn Response Selection in Retrieval-based Chatbots
share
We consider matching with pre-trained contextualized word vectors for multi-turn response selection in retrieval-based chatbots. When directly applied to the task, state-of-the-art models, such as CoVe and ELMo, do not work as well as they do on other tasks, due to the hierarchical nature, casual language, and domain-specific word use of conversations. To tackle the challenges, we propose pre-training a sentence-level and a session-level contextualized word vectors by learning a dialogue generation model from large-scale human-human conversations with a hierarchical encoder-decoder architecture. The two levels of vectors are then integrated into the input layer and the output layer of a matching model respectively. Experimental results on two benchmark datasets indicate that the proposed contextualized word vectors can significantly and consistently improve the performance of existing matching models for response selection.
READ FULL TEXT