TraVeLGAN: Image-to-image Translation by Transformation Vector Learning

02/25/2019
by   Matthew Amodio, et al.
36

Interest in image-to-image translation has grown substantially in recent years with the success of unsupervised models based on the cycle-consistency assumption. The achievements of these models have been limited to a particular subset of domains where this assumption yields good results, namely homogeneous domains that are characterized by style or texture differences. We tackle the challenging problem of image-to-image translation where the domains are defined by high-level shapes and contexts, as well as including significant clutter and heterogeneity. For this purpose, we introduce a novel GAN based on preserving intra-domain vector transformations in a latent space learned by a siamese network. The traditional GAN system introduced a discriminator network to guide the generator into generating images in the target domain. To this two-network system we add a third: a siamese network that guides the generator so that each original image shares semantics with its generated version. With this new three-network system, we no longer need to constrain the generators with the ubiquitous cycle-consistency restraint. As a result, the generators can learn mappings between more complex domains that differ from each other by large differences - not just style or texture.

READ FULL TEXT

page 1

page 2

page 4

page 5

page 7

page 12

research
06/23/2020

Image-to-image Mapping with Many Domains by Sparse Attribute Transfer

Unsupervised image-to-image translation consists of learning a pair of m...
research
10/21/2021

A Domain Gap Aware Generative Adversarial Network for Multi-domain Image Translation

Recent image-to-image translation models have shown great success in map...
research
11/09/2022

ParGAN: Learning Real Parametrizable Transformations

Current methods for image-to-image translation produce compelling result...
research
01/14/2019

XNet: GAN Latent Space Constraints

Recent GAN-based architectures have been able to deliver impressive perf...
research
10/11/2021

Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation

The image-to-image translation (I2IT) model takes a target label or a re...
research
12/09/2020

Lipschitz Regularized CycleGAN for Improving Semantic Robustness in Unpaired Image-to-image Translation

For unpaired image-to-image translation tasks, GAN-based approaches are ...
research
01/18/2021

Visualizing Missing Surfaces In Colonoscopy Videos using Shared Latent Space Representations

Optical colonoscopy (OC), the most prevalent colon cancer screening tool...

Please sign up or login with your details

Forgot password? Click here to reset