FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentation

by   Luca Sestini, et al.
Université de Strasbourg

Automatic surgical instrument segmentation of endoscopic images is a crucial building block of many computer-assistance applications for minimally invasive surgery. So far, state-of-the-art approaches completely rely on the availability of a ground-truth supervision signal, obtained via manual annotation, thus expensive to collect at large scale. In this paper, we present FUN-SIS, a Fully-UNsupervised approach for binary Surgical Instrument Segmentation. FUN-SIS trains a per-frame segmentation model on completely unlabelled endoscopic videos, by solely relying on implicit motion information and instrument shape-priors. We define shape-priors as realistic segmentation masks of the instruments, not necessarily coming from the same dataset/domain as the videos. The shape-priors can be collected in various and convenient ways, such as recycling existing annotations from other datasets. We leverage them as part of a novel generative-adversarial approach, allowing to perform unsupervised instrument segmentation of optical-flow images during training. We then use the obtained instrument masks as pseudo-labels in order to train a per-frame segmentation model; to this aim, we develop a learning-from-noisy-labels architecture, designed to extract a clean supervision signal from these pseudo-labels, leveraging their peculiar noise properties. We validate the proposed contributions on three surgical datasets, including the MICCAI 2017 EndoVis Robotic Instrument Segmentation Challenge dataset. The obtained fully-unsupervised results for surgical instrument segmentation are almost on par with the ones of fully-supervised state-of-the-art approaches. This suggests the tremendous potential of the proposed method to leverage the great amount of unlabelled data produced in the context of minimally invasive surgery.


page 2

page 5

page 9

page 10

page 12

page 19

page 23

page 24


Unsupervised Surgical Instrument Segmentation via Anchor Generation and Semantic Diffusion

Surgical instrument segmentation is a key component in developing contex...

SegMatch: A semi-supervised learning method for surgical instrument segmentation

Surgical instrument segmentation is recognised as a key enabler to provi...

SAF-IS: a Spatial Annotation Free Framework for Instance Segmentation of Surgical Tools

Instance segmentation of surgical instruments is a long-standing researc...

Simulation-to-Real domain adaptation with teacher-student learning for endoscopic instrument segmentation

Purpose: Segmentation of surgical instruments in endoscopic videos is es...

Please sign up or login with your details

Forgot password? Click here to reset