Rethinking Surgical Instrument Segmentation: A Background Image Can Be All You Need

by   An Wang, et al.
National University of Singapore
Imperial College London
The Chinese University of Hong Kong

Data diversity and volume are crucial to the success of training deep learning models, while in the medical imaging field, the difficulty and cost of data collection and annotation are especially huge. Specifically in robotic surgery, data scarcity and imbalance have heavily affected the model accuracy and limited the design and deployment of deep learning-based surgical applications such as surgical instrument segmentation. Considering this, we rethink the surgical instrument segmentation task and propose a one-to-many data generation solution that gets rid of the complicated and expensive process of data collection and annotation from robotic surgery. In our method, we only utilize a single surgical background tissue image and a few open-source instrument images as the seed images and apply multiple augmentations and blending techniques to synthesize amounts of image variations. In addition, we also introduce the chained augmentation mixing during training to further enhance the data diversities. The proposed approach is evaluated on the real datasets of the EndoVis-2018 and EndoVis-2017 surgical scene segmentation. Our empirical analysis suggests that without the high cost of data collection and annotation, we can achieve decent surgical instrument segmentation performance. Moreover, we also observe that our method can deal with novel instrument prediction in the deployment domain. We hope our inspiring results will encourage researchers to emphasize data-centric methods to overcome demanding deep learning limitations besides data shortage, such as class imbalance, domain adaptation, and incremental learning. Our code is available at


page 3

page 6


Generalizing Surgical Instruments Segmentation to Unseen Domains with One-to-Many Synthesis

Despite their impressive performance in various surgical scene understan...

Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning

Semantic segmentation of robotic instruments is an important problem for...

Simulation-to-Real domain adaptation with teacher-student learning for endoscopic instrument segmentation

Purpose: Segmentation of surgical instruments in endoscopic videos is es...

Endo-Sim2Real: Consistency learning-based domain adaptation for instrument segmentation

Surgical tool segmentation in endoscopic videos is an important componen...

Surgical Phase and Instrument Recognition: How to identify appropriate Dataset Splits

Purpose: The development of machine learning models for surgical workflo...

AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentation

Segmentation is a fundamental problem in surgical scene analysis using a...

Are You an Introvert or Extrovert? Accurate Classification With Only Ten Predictors

This paper investigates how accurately the prediction of being an introv...

Please sign up or login with your details

Forgot password? Click here to reset