Reducing Annotating Load: Active Learning with Synthetic Images in Surgical Instrument Segmentation

by   Haonan Peng, et al.

Accurate instrument segmentation in endoscopic vision of robot-assisted surgery is challenging due to reflection on the instruments and frequent contacts with tissue. Deep neural networks (DNN) show competitive performance and are in favor in recent years. However, the hunger of DNN for labeled data poses a huge workload of annotation. Motivated by alleviating this workload, we propose a general embeddable method to decrease the usage of labeled real images, using active generated synthetic images. In each active learning iteration, the most informative unlabeled images are first queried by active learning and then labeled. Next, synthetic images are generated based on these selected images. The instruments and backgrounds are cropped out and randomly combined with each other with blending and fusion near the boundary. The effectiveness of the proposed method is validated on 2 sinus surgery datasets and 1 intraabdominal surgery dataset. The results indicate a considerable improvement in performance, especially when the budget for annotation is small. The effectiveness of different types of synthetic images, blending methods, and external background are also studied. All the code is open-sourced at:


page 1

page 2

page 3

page 4

page 5

page 8


Towards Better Surgical Instrument Segmentation in Endoscopic Vision: Multi-Angle Feature Aggregation and Contour Supervision

Accurate and real-time surgical instrument segmentation is important in ...

Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning

Semantic segmentation of robotic instruments is an important problem for...

Generalizing Surgical Instruments Segmentation to Unseen Domains with One-to-Many Synthesis

Despite their impressive performance in various surgical scene understan...

Active Learning Based Domain Adaptation for Tissue Segmentation of Histopathological Images

Accurate segmentation of tissue in histopathological images can be very ...

Active Learning using Deep Bayesian Networks for Surgical Workflow Analysis

For many applications in the field of computer assisted surgery, such as...

Graph-based Reinforcement Learning for Active Learning in Real Time: An Application in Modeling River Networks

Effective training of advanced ML models requires large amounts of label...

Deep Active Learning by Model Interpretability

Recent successes of Deep Neural Networks (DNNs) in a variety of research...

Please sign up or login with your details

Forgot password? Click here to reset