Tactile-Sensitive NewtonianVAE for High-Accuracy Industrial Connector-Socket Insertion

by   Ryo Okumura, et al.

An industrial connector-socket insertion task requires sub-millimeter positioning and compensation of grasp pose of a connector. Thus high accurate estimation of relative pose between socket and connector is a key factor to achieve the task. World models are promising technology for visuo-motor control. They obtain appropriate state representation for control to jointly optimize feature extraction and latent dynamics model. Recent study shows NewtonianVAE, which is a kind of the world models, acquires latent space which is equivalent to mapping from images to physical coordinate. Proportional control can be achieved in the latent space of NewtonianVAE. However, application of NewtonianVAE to high accuracy industrial tasks in physical environments is open problem. Moreover, there is no general frameworks to compensate goal position in the obtained latent space considering the grasp pose. In this work, we apply NewtonianVAE to USB connector insertion with grasp pose variation in the physical environments. We adopt a GelSight type tactile sensor and estimate insertion position compensated by the grasp pose of the connector. Our method trains the latent space in an end-to-end manner, and simple proportional control is available. Therefore, it requires no additional engineering and annotation. Experimental results show that the proposed method, Tactile-Sensitive NewtonianVAE, outperforms naive combination of regression-based grasp pose estimator and coordinate transformation. Moreover, we reveal the original NewtonianVAE does not work in some situation, and demonstrate that domain knowledge induction improves model accuracy. This domain knowledge is easy to be known from specification of robots or measurement.


Center-of-Mass-based Robust Grasp Planning for Unknown Objects Using Tactile-Visual Sensors

An unstable grasp pose can lead to slip, thus an unstable grasp pose can...

Visuotactile Affordances for Cloth Manipulation with Local Control

Cloth in the real world is often crumpled, self-occluded, or folded in o...

Safely Learning Visuo-Tactile Feedback Policies in Real For Industrial Insertion

Industrial insertion tasks are often performed repetitively with parts t...

PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis

When humans grasp objects in the real world, we often move our arms to h...

Tactile Regrasp: Grasp Adjustments via Simulated Tactile Transformations

This paper presents a novel regrasp control policy that makes use of tac...

Learn Proportional Derivative Controllable Latent Space from Pixels

Recent advances in latent space dynamics model from pixels show promisin...

DeFNet: Deconstructed Fabric Folding Strategy Based on Latent Space Roadmap and Flow-Based Policy

Fabric folding through robots is complex and challenging due to the defo...

Please sign up or login with your details

Forgot password? Click here to reset