Economical Precise Manipulation and Auto Eye-Hand Coordination with Binocular Visual Reinforcement Learning

by   Yiwen Chen, et al.

Precision robotic manipulation tasks (insertion, screwing, precisely pick, precisely place) are required in many scenarios. Previous methods achieved good performance on such manipulation tasks. However, such methods typically require tedious calibration or expensive sensors. 3D/RGB-D cameras and torque/force sensors add to the cost of the robotic application and may not always be economical. In this work, we aim to solve these but using only weak-calibrated and low-cost webcams. We propose Binocular Alignment Learning (BAL), which could automatically learn the eye-hand coordination and points alignment capabilities to solve the four tasks. Our work focuses on working with unknown eye-hand coordination and proposes different ways of performing eye-in-hand camera calibration automatically. The algorithm was trained in simulation and used a practical pipeline to achieve sim2real and test it on the real robot. Our method achieves a competitively good result with minimal cost on the four tasks.


page 2

page 4

page 9


Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection

We describe a learning-based approach to hand-eye coordination for robot...

Applications of Uncalibrated Image Based Visual Servoing in Micro- and Macroscale Robotics

We present a robust markerless image based visual servoing method that e...

Solving the Robot-World Hand-Eye(s) Calibration Problem with Iterative Methods

Robot-world, hand-eye calibration is the problem of determining the tran...

Towards Robotic Eye Surgery: Marker-free, Online Hand-eye Calibration using Optical Coherence Tomography Images

Ophthalmic microsurgery is known to be a challenging operation, which re...

Reinforcement Learning of Active Vision forManipulating Objects under Occlusions

We consider artificial agents that learn to jointly control their grippe...

Evaluation of state representation methods in robot hand-eye coordination learning from demonstration

We evaluate different state representation methods in robot hand-eye coo...

Please sign up or login with your details

Forgot password? Click here to reset