Multi-Objective Convolutional Neural Networks for Robot Localisation and 3D Position Estimation in 2D Camera Images

by   Justinas Miseikis, et al.

The field of collaborative robotics and human-robot interaction often focuses on the prediction of human behaviour, while assuming the information about the robot setup and configuration being known. This is often the case with fixed setups, which have all the sensors fixed and calibrated in relation to the rest of the system. However, it becomes a limiting factor when the system needs to be reconfigured or moved. We present a deep learning approach, which aims to solve this issue. Our method learns to identify and precisely localise the robot in 2D camera images, so having a fixed setup is no longer a requirement and a camera can be moved. In addition, our approach identifies the robot type and estimates the 3D position of the robot base in the camera image as well as 3D positions of each of the robot joints. Learning is done by using a multi-objective convolutional neural network with four previously mentioned objectives simultaneously using a combined loss function. The multi-objective approach makes the system more flexible and efficient by reusing some of the same features and diversifying for each objective in lower layers. A fully trained system shows promising results in providing an accurate mask of where the robot is located and an estimate of its base and joint positions in 3D. We compare the results to our previous approach of using cascaded convolutional neural networks.


page 2

page 3

page 4

page 6


Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks

Many works in collaborative robotics and human-robot interaction focuses...

Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image using CNN

Collaborative robots are becoming more common on factory floors as well ...

Transfer Learning for Unseen Robot Detection and Joint Estimation on a Multi-Objective Convolutional Neural Network

A significant problem of using deep learning techniques is the limited a...

Anticipatory Human-Robot Collaboration via Multi-Objective Trajectory Optimization

We address the problem of adapting robot trajectories to improve safety,...

Toward Force Estimation in Robot-Assisted Surgery using Deep Learning with Vision and Robot State

Knowledge of interaction forces during teleoperated robot-assisted surge...

Poisson's CDF applied to Flexible Skylines

The evolution of skyline and ranking queries has created new archetypes ...

Please sign up or login with your details

Forgot password? Click here to reset