Learning to Simulate Tree-Branch Dynamics for Manipulation

by   Jayadeep Jacob, et al.
The University of Sydney

We propose to use a simulation driven inverse inference approach to model the joint dynamics of tree branches under manipulation. Learning branch dynamics and gaining the ability to manipulate deformable vegetation can help with occlusion-prone tasks, such as fruit picking in dense foliage, as well as moving overhanging vines and branches for navigation in dense vegetation. The underlying deformable tree geometry is encapsulated as coarse spring abstractions executed on parallel, non-differentiable simulators. The implicit statistical model defined by the simulator, reference trajectories obtained by actively probing the ground truth, and the Bayesian formalism, together guide the spring parameter posterior density estimation. Our non-parametric inference algorithm, based on Stein Variational Gradient Descent, incorporates biologically motivated assumptions into the inference process as neural network driven learnt joint priors; moreover, it leverages the finite difference scheme for gradient approximations. Real and simulated experiments confirm that our model can predict deformation trajectories, quantify the estimation uncertainty, and it can perform better when base-lined against other inference algorithms, particularly from the Monte Carlo family. The model displays strong robustness properties in the presence of heteroscedastic sensor noise; furthermore, it can generalise to unseen grasp locations.


page 1

page 3

page 7


Robust Shape Estimation for 3D Deformable Object Manipulation

Existing shape estimation methods for deformable object manipulation suf...

A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation

Deformable object manipulation remains a challenging task in robotics re...

A real-time, hardware agnostic framework for close-up branch reconstruction using RGB data

Creating accurate 3D models of tree topology is an important task for tr...

Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulation

To accurately reproduce measurements from the real world, simulators nee...

GenORM: Generalizable One-shot Rope Manipulation with Parameter-Aware Policy

Due to the inherent uncertainty in their deformability during motion, pr...

Recreating Bat Behavior on Quad-rotor UAVs-A Simulation Approach

We develop an effective computer model to simulate sensing environments ...

Please sign up or login with your details

Forgot password? Click here to reset