Sim-to-Real Segmentation in Robot-assisted Transoral Tracheal Intubation

by   Guankun Wang, et al.

Robotic-assisted tracheal intubation requires the robot to distinguish anatomical features like an experienced physician using deep-learning techniques. However, real datasets of oropharyngeal organs are limited due to patient privacy issues, making it challenging to train deep-learning models for accurate image segmentation. We hereby consider generating a new data modality through a virtual environment to assist the training process. Specifically, this work introduces a virtual dataset generated by the Simulation Open Framework Architecture (SOFA) framework to overcome the limited availability of actual endoscopic images. We also propose a domain adaptive Sim-to-Real method for oropharyngeal organ image segmentation, which employs an image blending strategy called IoU-Ranking Blend (IRB) and style-transfer techniques to address discrepancies between datasets. Experimental results demonstrate the superior performance of the proposed approach with domain adaptive models, improving segmentation accuracy and training stability. In the practical application, the trained segmentation model holds great promise for robot-assisted intubation surgery and intelligent surgical navigation.


page 1

page 2


Domain Adaptive Sim-to-Real Segmentation of Oropharyngeal Organs

Video-assisted transoral tracheal intubation (TI) necessitates using an ...

Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data

Surgical instrument segmentation for robot-assisted surgery is needed fo...

Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery

Semantic tool segmentation in surgical videos is important for surgical ...

Segmentation of kidney stones in endoscopic video feeds

Image segmentation has been increasingly applied in medical settings as ...

Deep Learning of Crystalline Defects from TEM images: A Solution for the Problem of "Never Enough Training Data"

Crystalline defects, such as line-like dislocations, play an important r...

Please sign up or login with your details

Forgot password? Click here to reset