Exploring Semantic Consistency in Unpaired Image Translation to Generate Data for Surgical Applications

by   Danush Kumar Venkatesh, et al.

In surgical computer vision applications, obtaining labeled training data is challenging due to data-privacy concerns and the need for expert annotation. Unpaired image-to-image translation techniques have been explored to automatically generate large annotated datasets by translating synthetic images to the realistic domain. However, preserving the structure and semantic consistency between the input and translated images presents significant challenges, mainly when there is a distributional mismatch in the semantic characteristics of the domains. This study empirically investigates unpaired image translation methods for generating suitable data in surgical applications, explicitly focusing on semantic consistency. We extensively evaluate various state-of-the-art image translation models on two challenging surgical datasets and downstream semantic segmentation tasks. We find that a simple combination of structural-similarity loss and contrastive learning yields the most promising results. Quantitatively, we show that the data generated with this approach yields higher semantic consistency and can be used more effectively as training data.


page 1

page 6

page 7

page 8

page 12

page 13

page 14


Unpaired Image-to-Image Translation using Adversarial Consistency Loss

Unpaired image-to-image translation is a class of vision problems whose ...

Unpaired Image Translation via Vector Symbolic Architectures

Image-to-image translation has played an important role in enabling synt...

Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Tasks

Recently, contrastive learning-based image translation methods have been...

Closing the Reality Gap with Unsupervised Sim-to-Real Image Translation for Semantic Segmentation in Robot Soccer

Deep learning approaches have become the standard solution to many probl...

Improving Endoscopic Decision Support Systems by Translating Between Imaging Modalities

Novel imaging technologies raise many questions concerning the adaptatio...

Leveraging in-domain supervision for unsupervised image-to-image translation tasks via multi-stream generators

Supervision for image-to-image translation (I2I) tasks is hard to come b...

Content-Preserving Unpaired Translation from Simulated to Realistic Ultrasound Images

Interactive simulation of ultrasound imaging greatly facilitates sonogra...

Please sign up or login with your details

Forgot password? Click here to reset