Leveraging Multi-modal Sensing for Robotic Insertion Tasks in R D Laboratories

by   Aaron Butterworth, et al.

Performing a large volume of experiments in Chemistry labs creates repetitive actions costing researchers time, automating these routines is highly desirable. Previous experiments in robotic chemistry have performed high numbers of experiments autonomously, however, these processes rely on automated machines in all stages from solid or liquid addition to analysis of the final product. In these systems every transition between machine requires the robotic chemist to pick and place glass vials, however, this is currently performed using open loop methods which require all equipment being used by the robot to be in well defined known locations. We seek to begin closing the loop in this vial handling process in a way which also fosters human-robot collaboration in the chemistry lab environment. To do this the robot must be able to detect valid placement positions for the vials it is collecting, and reliably insert them into the detected locations. We create a single modality visual method for estimating placement locations to provide a baseline before introducing two additional methods of feedback (force and tactile feedback). Our visual method uses a combination of classic computer vision methods and a CNN discriminator to detect possible insertion points, then a vial is grasped and positioned above an insertion point and the multi-modal methods guide the final insertion movements using an efficient search pattern. Through our experiments we show the baseline insertion rate of 48.78 our "force and vision" multi-modal feedback method.


page 1

page 2

page 3


Rotating Objects via In-Hand Pivoting using Vision, Force and Touch

We propose a robotic manipulation system that can pivot objects on a sur...

Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning

Combined visual and force feedback play an essential role in contact-ric...

Using The Feedback of Dynamic Active-Pixel Vision Sensor (Davis) to Prevent Slip in Real Time

The objective of this paper is to describe an approach to detect the sli...

Collaborative Robotic Biopsy with Trajectory Guidance and Needle Tip Force Feedback

The diagnostic value of biopsies is highly dependent on the placement of...

Combining Vision and Tactile Sensation for Video Prediction

In this paper, we explore the impact of adding tactile sensation to vide...

DroneStick: Flying Joystick as a Novel Type of Interface

DroneStick is a novel hands-free method for smooth interaction between a...

Hybrid Open-Loop Closed-Loop Control of Coupled Human-Robot Balance During Assisted Stance Transition with Extra Robotic Legs

A new approach to the human-robot shared control of the Extra Robotic Le...

Please sign up or login with your details

Forgot password? Click here to reset