Real-Time Instrument Segmentation in Robotic Surgery using Auxiliary Supervised Deep Adversarial Learning

by   Mobarakol Islam, et al.

Robot-assisted surgery is an emerging technology which has undergone rapid growth with the development of robotics and imaging systems. Innovations in vision, haptics and accurate movements of robot arms have enabled surgeons to perform precise minimally invasive surgeries. Real-time semantic segmentation of the robotic instruments and tissues is a crucial step in robot-assisted surgery. Accurate and efficient segmentation of the surgical scene not only aids in the identification and tracking of instruments but also provided contextual information about the different tissues and instruments being operated with. For this purpose, we have developed a light-weight cascaded convolutional neural network (CNN) to segment the surgical instruments from high-resolution videos obtained from a commercial robotic system. We propose a multi-resolution feature fusion module (MFF) to fuse the feature maps of different dimensions and channels from the auxiliary and main branch. We also introduce a novel way of combining auxiliary loss and adversarial loss to regularize the segmentation model. Auxiliary loss helps the model to learn low-resolution features, and adversarial loss improves the segmentation prediction by learning higher order structural information. The model also consists of a light-weight spatial pyramid pooling (SPP) unit to aggregate rich contextual information in the intermediate stage. We show that our model surpasses existing algorithms for pixel-wise segmentation of surgical instruments in both prediction accuracy and segmentation time of high-resolution videos.


page 1

page 2

page 4

page 6

page 7


Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery

Segmentation of surgical instruments is an important problem in robot-as...

Attention-Guided Lightweight Network for Real-Time Segmentation of Robotic Surgical Instruments

Real-time segmentation of surgical instruments plays a crucial role in r...

RASNet: Segmentation for Tracking Surgical Instruments in Surgical Videos Using Refined Attention Segmentation Network

Segmentation for tracking surgical instruments plays an important role i...

Learning Where to Look While Tracking Instruments in Robot-assisted Surgery

Directing of the task-specific attention while tracking instrument in su...

A Nested U-Structure for Instrument Segmentation in Robotic Surgery

Robot-assisted surgery has made great progress with the development of m...

ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools

Real-time tool segmentation from endoscopic videos is an essential part ...

Learning Domain Adaptation with Model Calibration for Surgical Report Generation in Robotic Surgery

Generating a surgical report in robot-assisted surgery, in the form of n...

Please sign up or login with your details

Forgot password? Click here to reset