Flexible Networks for Learning Physical Dynamics of Deformable Objects

by   Jinhyung Park, et al.

Learning the physical dynamics of deformable objects with particle-based representation has been the objective of many computational models in machine learning. While several state-of-the-art models have achieved this objective in simulated environments, most existing models impose a precondition, such that the input is a sequence of ordered point sets - i.e., the order of the points in each point set must be the same across the entire input sequence. This restrains the model to generalize to real-world data, which is considered to be a sequence of unordered point sets. In this paper, we propose a model named time-wise PointNet (TP-Net) that solves this problem by directly consuming a sequence of unordered point sets to infer the future state of a deformable object with particle-based representation. Our model consists of a shared feature extractor that extracts global features from each input point set in parallel and a prediction network that aggregates and reasons on these features for future prediction. The key concept of our approach is that we use global features rather than local features to achieve invariance to input permutations and ensure the stability and scalability of our model. Experiments demonstrate that our model achieves state-of-the-art performance in both synthetic dataset and in real-world dataset, with real-time prediction speed. We provide quantitative and qualitative analysis on why our approach is more effective and efficient than existing approaches.


page 5

page 7

page 10


EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics

We study the problem of learning graph dynamics of deformable objects wh...

Visual Grounding of Learned Physical Models

Humans intuitively recognize objects' physical properties and predict th...

AGAR: Attention Graph-RNN for Adaptative Motion Prediction of Point Clouds of Deformable Objects

This paper focuses on motion prediction for point cloud sequences in the...

Global-Local Aggregation with Deformable Point Sampling for Camouflaged Object Detection

The camouflaged object detection (COD) task aims to find and segment obj...

VIRDO++: Real-World, Visuo-tactile Dynamics and Perception of Deformable Objects

Deformable objects manipulation can benefit from representations that se...

Sequential Topological Representations for Predictive Models of Deformable Objects

Deformable objects present a formidable challenge for robotic manipulati...

Learning to Represent and Predict Sets with Deep Neural Networks

In this thesis, we develop various techniques for working with sets in m...

Please sign up or login with your details

Forgot password? Click here to reset