Human-Robot Motion Retargeting via Neural Latent Optimization

by   Haodong Zhang, et al.

Motion retargeting from human to robot remains a very challenging task due to variations in the structure of humans and robots. Most traditional optimization-based algorithms solve this problem by minimizing an objective function, which is usually time-consuming and heavily dependent on good initialization. In contrast, methods with feedforward neural networks can learn prior knowledge from training data and quickly infer the results, but these methods also suffer from the generalization problem on unseen actions, leading to some infeasible results. In this paper, we propose a novel neural optimization approach taking advantages of both kinds of methods. A graph-based neural network is utilized to establish a mapping between the latent space and the robot motion space. Afterward, the retargeting results can be obtained by searching for the optimal vector in this latent space. In addition, a deep encoder also provides a promising initialization for better and faster convergence. We perform experiments on retargeting Chinese sign language to three different kinds of robots in the simulation environment, including ABB's YuMi dual-arm collaborative robot, NAO and Pepper. A real-world experiment is also conducted on the Yumi robot. Experimental results show that our method can retarget motion from human to robot with both efficiency and accuracy.


page 1

page 6

page 7

page 8


Planning Coordinated Human-Robot Motions with Neural Network Full-Body Prediction Models

Numerical optimization has become a popular approach to plan smooth moti...

Unsupervised human-to-robot motion retargeting via expressive latent space

This paper introduces a novel approach for human-to-robot motion retarge...

Dynamic Movement Primitive based Motion Retargeting for Dual-Arm Sign Language Motions

We aim to develop an efficient programming method for equipping service ...

Latent Emission-Augmented Perspective-Taking (LEAPT) for Human-Robot Interaction

Perspective-taking is the ability to perceive or understand a situation ...

C-3PO: Cyclic-Three-Phase Optimization for Human-Robot Motion Retargeting based on Reinforcement Learning

Motion retargeting between heterogeneous polymorphs with different sizes...

Transferability-based Chain Motion Mapping from Humans to Humanoids for Teleoperation

Although data-driven motion mapping methods are promising to allow intui...

Human-to-Robot Attention Transfer for Robot Execution Failure Avoidance Using Stacked Neural Networks

Due to world dynamics and hardware uncertainty, robots inevitably fail i...

Please sign up or login with your details

Forgot password? Click here to reset