SAF-IS: a Spatial Annotation Free Framework for Instance Segmentation of Surgical Tools

by   Luca Sestini, et al.
Université de Strasbourg

Instance segmentation of surgical instruments is a long-standing research problem, crucial for the development of many applications for computer-assisted surgery. This problem is commonly tackled via fully-supervised training of deep learning models, requiring expensive pixel-level annotations to train. In this work, we develop a framework for instance segmentation not relying on spatial annotations for training. Instead, our solution only requires binary tool masks, obtainable using recent unsupervised approaches, and binary tool presence labels, freely obtainable in robot-assisted surgery. Based on the binary mask information, our solution learns to extract individual tool instances from single frames, and to encode each instance into a compact vector representation, capturing its semantic features. Such representations guide the automatic selection of a tiny number of instances (8 only in our experiments), displayed to a human operator for tool-type labelling. The gathered information is finally used to match each training instance with a binary tool presence label, providing an effective supervision signal to train a tool instance classifier. We validate our framework on the EndoVis 2017 and 2018 segmentation datasets. We provide results using binary masks obtained either by manual annotation or as predictions of an unsupervised binary segmentation model. The latter solution yields an instance segmentation approach completely free from spatial annotations, outperforming several state-of-the-art fully-supervised segmentation approaches.


Training-Free Instance Segmentation from Semantic Image Segmentation Masks

In recent years, the development of instance segmentation has garnered s...

UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes

3D instance segmentation is fundamental to geometric understanding of th...

Deep-learning in the bioimaging wild: Handling ambiguous data with deepflash2

We present deepflash2, a deep learning solution that facilitates the obj...

FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentation

Automatic surgical instrument segmentation of endoscopic images is a cru...

Vision Transformers Are Good Mask Auto-Labelers

We propose Mask Auto-Labeler (MAL), a high-quality Transformer-based mas...

Where are the Masks: Instance Segmentation with Image-level Supervision

A major obstacle in instance segmentation is that existing methods often...

Incremental Learning on Food Instance Segmentation

Food instance segmentation is essential to estimate the serving size of ...

Please sign up or login with your details

Forgot password? Click here to reset