Towards a Human-like Open-Domain Chatbot

01/27/2020
by   Daniel Adiwardana, et al.
16

We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is trained to minimize perplexity, an automatic metric that we compare against human judgement of multi-turn conversation quality. To capture this judgement, we propose a human evaluation metric called Sensibleness and Specificity Average (SSA), which captures key elements of good conversation. Interestingly, our experiments show strong correlation between perplexity and SSA. The fact that the best perplexity end-to-end trained Meena scores high on SSA (72 SSA of 86 Additionally, the full version of Meena (with a filtering mechanism and tuned decoding) scores 79 we evaluated.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset