TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting

by   Zhuoqian Yang, et al.

We present a lightweight video motion retargeting approach TransMoMo that is capable of transferring motion of a person in a source video realistically to another video of a target person. Without using any paired data for supervision, the proposed method can be trained in an unsupervised manner by exploiting invariance properties of three orthogonal factors of variation including motion, structure, and view-angle. Specifically, with loss functions carefully derived based on invariance, we train an auto-encoder to disentangle the latent representations of such factors given the source and target video clips. This allows us to selectively transfer motion extracted from the source video seamlessly to the target video in spite of structural and view-angle disparities between the source and the target. The relaxed assumption of paired data allows our method to be trained on a vast amount of videos needless of manual annotation of source-target pairing, leading to improved robustness against large structural variations and extreme motion in videos. We demonstrate the effectiveness of our method over the state-of-the-art methods. Code, model and data are publicly available on our project page (


page 1

page 3

page 6

page 7

page 8

page 12


Do as we do: Multiple Person Video-To-Video Transfer

Our goal is to transfer the motion of real people from a source video to...

Everybody Dance Now

This paper presents a simple method for "do as I do" motion transfer: gi...

StyleMask: Disentangling the Style Space of StyleGAN2 for Neural Face Reenactment

In this paper we address the problem of neural face reenactment, where, ...

Target Consistency for Domain Adaptation: when Robustness meets Transferability

Learning Invariant Representations has been successfully applied for rec...

Memory Efficient Temporal Visual Graph Model for Unsupervised Video Domain Adaptation

Existing video domain adaption (DA) methods need to store all temporal c...

Motion-Based Weak Supervision for Video Parsing with Application to Colonoscopy

We propose a two-stage unsupervised approach for parsing videos into pha...

Novel View Synthesis on Unpaired Data by Conditional Deformable Variational Auto-Encoder

Novel view synthesis often needs the paired data from both the source an...

Please sign up or login with your details

Forgot password? Click here to reset