Learning Where to Look While Tracking Instruments in Robot-assisted Surgery

by   Mobarakol Islam, et al.
National University of Singapore

Directing of the task-specific attention while tracking instrument in surgery holds great potential in robot-assisted intervention. For this purpose, we propose an end-to-end trainable multitask learning (MTL) model for real-time surgical instrument segmentation and attention prediction. Our model is designed with a weight-shared encoder and two task-oriented decoders and optimized for the joint tasks. We introduce batch-Wasserstein (bW) loss and construct a soft attention module to refine the distinctive visual region for efficient saliency learning. For multitask optimization, it is always challenging to obtain convergence of both tasks in the same epoch. We deal with this problem by adopting `poly' loss weight and two phases of training. We further propose a novel way to generate task-aware saliency map and scanpath of the instruments on MICCAI robotic instrument segmentation dataset. Compared to the state of the art segmentation and saliency models, our model outperforms most of the evaluation metrics.


page 3

page 5

page 6

page 7


Real-Time Instrument Segmentation in Robotic Surgery using Auxiliary Supervised Deep Adversarial Learning

Robot-assisted surgery is an emerging technology which has undergone rap...

Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery

Intraoperative segmentation and tracking of minimally invasive instrumen...

Attention-Guided Lightweight Network for Real-Time Segmentation of Robotic Surgical Instruments

Real-time segmentation of surgical instruments plays a crucial role in r...

Real-time Joint Motion Analysis and Instrument Tracking for Robot-Assisted Orthopaedic Surgery

Robotic-assisted orthopaedic surgeries demand accurate, automated leg ma...

Please sign up or login with your details

Forgot password? Click here to reset