Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions

by   Weilin Wan, et al.

Understanding human intentions during interactions has been a long-lasting theme, that has applications in human-robot interaction, virtual reality and surveillance. In this study, we focus on full-body human interactions with large-sized daily objects and aim to predict the future states of objects and humans given a sequential observation of human-object interaction. As there is no such dataset dedicated to full-body human interactions with large-sized daily objects, we collected a large-scale dataset containing thousands of interactions for training and evaluation purposes. We also observe that an object's intrinsic physical properties are useful for the object motion prediction, and thus design a set of object dynamic descriptors to encode such intrinsic properties. We treat the object dynamic descriptors as a new modality and propose a graph neural network, HO-GCN, to fuse motion data and dynamic descriptors for the prediction task. We show the proposed network that consumes dynamic descriptors can achieve state-of-the-art prediction results and help the network better generalize to unseen objects. We also demonstrate the predicted results are useful for human-robot collaborations.


page 1

page 3

page 4

page 7


BEHAVE: Dataset and Method for Tracking Human Object Interactions

Modelling interactions between humans and objects in natural environment...

Object Properties Inferring from and Transfer for Human Interaction Motions

Humans regularly interact with their surrounding objects. Such interacti...

Probabilistic Human Motion Prediction via A Bayesian Neural Network

Human motion prediction is an important and challenging topic that has p...

Dynamic Robot Motion Prediction Updates in Physical Human-Robot Interactive Tasks

Human-robot collaboration is on the rise. Robots need to increasingly im...

Context-aware Human Motion Prediction

The problem of predicting human motion given a sequence of past observat...

Human Hands as Probes for Interactive Object Understanding

Interactive object understanding, or what we can do to objects and how i...

PIP: Physical Interaction Prediction via Mental Imagery with Span Selection

To align advanced artificial intelligence (AI) with human values and pro...

Please sign up or login with your details

Forgot password? Click here to reset