Learning Fine Pinch-Grasp Skills using Tactile Sensing from Real Demonstration Data

by   Xiaofeng Mao, et al.

This work develops a data-efficient learning from demonstration framework which exploits the use of rich tactile sensing and achieves fine dexterous bimanual manipulation. Specifically, we formulated a convolutional autoencoder network that can effectively extract and encode high-dimensional tactile information. Further, we developed a behaviour cloning network that can learn human-like sensorimotor skills demonstrated directly on the robot hardware in the task space by fusing both proprioceptive and tactile feedback. Our comparison study with the baseline method revealed the effectiveness of the contact information, which enabled successful extraction and replication of the demonstrated motor skills. Extensive experiments on real dual-arm robots demonstrated the robustness and effectiveness of the fine pinch grasp policy directly learned from one-shot demonstration, including grasping of the same object with different initial poses, generalizing to ten unseen new objects, robust and firm grasping against external pushes, as well as contact-aware and reactive re-grasping in case of dropping objects under very large perturbations. Moreover, the saliency map method is employed to describe the weight distribution across various modalities during pinch grasping. The video is available online at: \href{https://youtu.be/4Pg29bUBKqs}{https://youtu.be/4Pg29bUBKqs}.


page 1

page 3

page 5

page 6

page 7


A robust controller for stable 3D pinching using tactile sensing

This paper proposes a controller for stable grasping of unknown-shaped o...

Slip detection for grasp stabilisation with a multi-fingered tactile robot hand

Tactile sensing is used by humans when grasping to prevent us dropping o...

Task-grasping from human demonstration

A challenge in robot grasping is to achieve task-grasping which is to se...

Fusing Visuo-Tactile Perception into Kernelized Synergies for Robust Grasping and Fine Manipulation of Non-rigid Objects

Handling non-rigid objects using robot hands necessities a framework tha...

Cherry-Picking with Reinforcement Learning

Grasping small objects surrounded by unstable or non-rigid material play...

Grasping Unknown Objects with Proprioception Using a Series-Elastic-Actuated Gripper

Grasping unknown objects has been an active research topic for decades. ...

Towards Robust Autonomous Grasping with Reflexes Using High-Bandwidth Sensing and Actuation

Modern robotic manipulation systems fall short of human manipulation ski...

Please sign up or login with your details

Forgot password? Click here to reset