Learning and Evaluating Human Preferences for Conversational Head Generation

by   Mohan Zhou, et al.
Harbin Institute of Technology

A reliable and comprehensive evaluation metric that aligns with manual preference assessments is crucial for conversational head video synthesis method development. Existing quantitative evaluations often fail to capture the full complexity of human preference, as they only consider limited evaluation dimensions. Qualitative evaluations and user studies offer a solution but are time-consuming and labor-intensive. This limitation hinders the advancement of conversational head generation algorithms and systems. In this paper, we propose a novel learning-based evaluation metric named Preference Score (PS) for fitting human preference according to the quantitative evaluations across different dimensions. PS can serve as a quantitative evaluation without the need for human annotation. Experimental results validate the superiority of Preference Score in aligning with human perception, and also demonstrates robustness and generalizability to unseen data, making it a valuable tool for advancing conversation head generation. We expect this metric could facilitate new advances in conversational head generation.


page 1

page 2

page 3

page 4


Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems

The lack of time-efficient and reliable evaluation methods hamper the de...

An Evaluation Protocol for Generative Conversational Systems

There is a multitude of novel generative models for open-domain conversa...

Interactive Conversational Head Generation

We introduce a new conversation head generation benchmark for synthesizi...

Perceptual Conversational Head Generation with Regularized Driver and Enhanced Renderer

This paper reports our solution for MultiMedia ViCo 2022 Conversational ...

Meta-evaluation of Conversational Search Evaluation Metrics

Conversational search systems, such as Google Assistant and Microsoft Co...

Are Large Language Models Good Evaluators for Abstractive Summarization?

Human evaluations are often required for abstractive summary evaluations...

What comprises a good talking-head video generation?: A Survey and Benchmark

Over the years, performance evaluation has become essential in computer ...

Please sign up or login with your details

Forgot password? Click here to reset