Visual-tactile Fusion for Transparent Object Grasping in Complex Backgrounds

by   Shoujie Li, et al.

The accurate detection and grasping of transparent objects are challenging but of significance to robots. Here, a visual-tactile fusion framework for transparent object grasping under complex backgrounds and variant light conditions is proposed, including the grasping position detection, tactile calibration, and visual-tactile fusion based classification. First, a multi-scene synthetic grasping dataset generation method with a Gaussian distribution based data annotation is proposed. Besides, a novel grasping network named TGCNN is proposed for grasping position detection, showing good results in both synthetic and real scenes. In tactile calibration, inspired by human grasping, a fully convolutional network based tactile feature extraction method and a central location based adaptive grasping strategy are designed, improving the success rate by 36.7 visual-tactile fusion method is proposed for transparent objects classification, which improves the classification accuracy by 34 framework synergizes the advantages of vision and touch, and greatly improves the grasping efficiency of transparent objects.


page 7

page 8

page 9

page 10

page 14

page 15

page 17

page 19


Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping

Picking up transparent objects is still a challenging task for robots. T...

Grasping in the Dark: Compliant Grasping using Shadow Dexterous Hand and BioTac Tactile Sensor

When it comes to grasping and manipulating objects, the human hand is th...

Multi-Modal Geometric Learning for Grasping and Manipulation

This work provides an architecture that incorporates depth and tactile i...

Fast robust peg-in-hole insertion with continuous visual servoing

This paper demonstrates a visual servoing method which is robust towards...

Grasping Using Tactile Sensing and Deep Calibration

Tactile perception is an essential ability of intelligent robots in inte...

Two-Stage Grasping: A New Bin Picking Framework for Small Objects

This paper proposes a novel bin picking framework, two-stage grasping, a...

Grasping Unknown Objects with Proprioception Using a Series-Elastic-Actuated Gripper

Grasping unknown objects has been an active research topic for decades. ...

Please sign up or login with your details

Forgot password? Click here to reset