Expert and Crowd-Guided Affect Annotation and Prediction
share
We employ crowdsourcing to acquire time-continuous affective annotations for movie clips, and refine noisy models trained from these crowd annotations incorporating expert information within a Multi-task Learning (MTL) framework. We propose a novel expert guided MTL (EG-MTL) algorithm, which minimizes the loss with respect to both crowd and expert labels to learn a set of weights corresponding to each movie clip for which crowd annotations are acquired. We employ EG-MTL to solve two problems, namely, : where dynamic annotations acquired from both experts and crowdworkers for the Validation set are used to train a regression model with audio-visual clip descriptors as features, and predict dynamic arousal and valence levels on 5–15 second snippets derived from the clips; and : where a classification model trained on the Validation set using dynamic crowd and expert annotations (as features) and static affective clip labels is used for binary emotion recognition on the Evaluation set for which only dynamic crowd annotations are available. Observed experimental results confirm the effectiveness of the EG-MTL algorithm, which is reflected via improved arousal and valence estimation for , and higher recognition accuracy for .
READ FULL TEXT