GAN-based Reactive Motion Synthesis with Class-aware Discriminators for Human-human Interaction

by   Qianhui Men, et al.

Creating realistic characters that can react to the users' or another character's movement can benefit computer graphics, games and virtual reality hugely. However, synthesizing such reactive motions in human-human interactions is a challenging task due to the many different ways two humans can interact. While there are a number of successful researches in adapting the generative adversarial network (GAN) in synthesizing single human actions, there are very few on modelling human-human interactions. In this paper, we propose a semi-supervised GAN system that synthesizes the reactive motion of a character given the active motion from another character. Our key insights are two-fold. First, to effectively encode the complicated spatial-temporal information of a human motion, we empower the generator with a part-based long short-term memory (LSTM) module, such that the temporal movement of different limbs can be effectively modelled. We further include an attention module such that the temporal significance of the interaction can be learned, which enhances the temporal alignment of the active-reactive motion pair. Second, as the reactive motion of different types of interactions can be significantly different, we introduce a discriminator that not only tells if the generated movement is realistic or not, but also tells the class label of the interaction. This allows the use of such labels in supervising the training of the generator. We experiment with the SBU and the HHOI datasets. The high quality of the synthetic motion demonstrates the effective design of our generator, and the discriminability of the synthesis also demonstrates the strength of our discriminator.


page 1

page 2

page 3

page 4


Synthesizing Physical Character-Scene Interactions

Movement is how people interact with and affect their environment. For r...

Interaction Transformer for Human Reaction Generation

We address the challenging task of human reaction generation which aims ...

Adversarial Attention for Human Motion Synthesis

Analysing human motions is a core topic of interest for many disciplines...

Generative Adversarial Network for Handwritten Text

Generative adversarial networks (GANs) has proven hugely successful in v...

Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction

Predicting an interaction before it is fully executed is very important ...

Human Motion Transfer with 3D Constraints and Detail Enhancement

We propose a new method for realistic human motion transfer using a gene...

AnimGAN: A Spatiotemporally-Conditioned Generative Adversarial Network for Character Animation

Producing realistic character animations is one of the essential tasks i...

Please sign up or login with your details

Forgot password? Click here to reset