UserSimCRS: A User Simulation Toolkit for Evaluating Conversational Recommender Systems

01/13/2023

∙

by   Jafar Afzali, et al.

∙

0

∙

share

We present an extensible user simulation toolkit to facilitate automatic evaluation of conversational recommender systems. It builds on an established agenda-based approach and extends it with several novel elements, including user satisfaction prediction, persona and context modeling, and conditional natural language generation. We showcase the toolkit with a pre-existing movie recommender system and demonstrate its ability to simulate dialogues that mimic real conversations, while requiring only a handful of manually annotated dialogues as training data.