Self-Supervised Surgical Tool Segmentation using Kinematic Information

by   Cristian da Costa Rocha, et al.
Université de Strasbourg

Surgical tool segmentation in endoscopic images is the first step towards pose estimation and (sub-)task automation in challenging minimally invasive surgical operations. While many approaches in the literature have shown great results using modern machine learning methods such as convolutional neural networks, the main bottleneck lies in the acquisition of a large number of manually-annotated images for efficient learning. This is especially true in surgical context, where patient-to-patient differences impede the overall generalizability. In order to cope with this lack of annotated data, we propose a self-supervised approach in a robot-assisted context. To our knowledge, the proposed approach is the first to make use of the kinematic model of the robot in order to generate training labels. The core contribution of the paper is to propose an optimization method to obtain good labels for training despite an unknown hand-eye calibration and an imprecise kinematic model. The labels can subsequently be used for fine-tuning a fully-convolutional neural network for pixel-wise classification. As a result, the tool can be segmented in the endoscopic images without needing a single manually-annotated image. Experimental results on phantom and in vivo datasets obtained using a flexible robotized endoscopy system are very promising.


page 1

page 2

page 3

page 6


A Kinematic Bottleneck Approach For Pose Regression of Flexible Surgical Instruments directly from Images

3-D pose estimation of instruments is a crucial step towards automatic s...

Multi-stage Suture Detection for Robot Assisted Anastomosis based on Deep Learning

In robotic surgery, task automation and learning from demonstration comb...

Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos

Surgical tool localization is an essential task for the automatic analys...

Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations

To improve the efficiency of surgical trajectory segmentation for robot ...

Self-supervised Skull Reconstruction in Brain CT Images with Decompressive Craniectomy

Decompressive craniectomy (DC) is a common surgical procedure consisting...

Please sign up or login with your details

Forgot password? Click here to reset