Generative Transition Mechanism to Image-to-Image Translation via Encoded Transformation

by   Yaxin Shi, et al.

In this paper, we revisit the Image-to-Image (I2I) translation problem with transition consistency, namely the consistency defined on the conditional data mapping between each data pairs. Explicitly parameterizing each data mappings with a transition variable t, i.e., x t(x,y)↦y, we discover that existing I2I translation models mainly focus on maintaining consistency on results, e.g., image reconstruction or attribute prediction, named result consistency in our paper. This restricts their generalization ability to generate satisfactory results with unseen transitions in the test phase. Consequently, we propose to enforce both result consistency and transition consistency for I2I translation, to benefit the problem with a closer consistency between the input and output. To benefit the generalization ability of the translation model, we propose transition encoding to facilitate explicit regularization of these two kinds of consistencies on unseen transitions. We further generalize such explicitly regularized consistencies to distribution-level, thus facilitating a generalized overall consistency for I2I translation problems. With the above design, our proposed model, named Transition Encoding GAN (TEGAN), can poss superb generalization ability to generate realistic and semantically consistent translation results with unseen transitions in the test phase. It also provides a unified understanding of the existing GAN-based I2I transition models with our explicitly modeling of the data mapping, i.e., transition. Experiments on four different I2I translation tasks demonstrate the efficacy and generality of TEGAN.


page 2

page 5

page 6

page 7

page 8


UTSGAN: Unseen Transition Suss GAN for Transition-Aware Image-to-image Translation

In the field of Image-to-Image (I2I) translation, ensuring consistency b...

Multiple GAN Inversion for Exemplar-based Image-to-Image Translation

Existing state-of-the-art techniques in exemplar-based image-to-image tr...

Augmented Cyclic Consistency Regularization for Unpaired Image-to-Image Translation

Unpaired image-to-image (I2I) translation has received considerable atte...

Toward Multimodal Image-to-Image Translation

Many image-to-image translation problems are ambiguous, as a single inpu...

Robustness via Uncertainty-aware Cycle Consistency

Unpaired image-to-image translation refers to learning inter-image-domai...

On Breiman's Dilemma in Neural Networks: Phase Transitions of Margin Dynamics

Margin enlargement over training data has been an important strategy sin...

Uncertainty-aware Generalized Adaptive CycleGAN

Unpaired image-to-image translation refers to learning inter-image-domai...

Please sign up or login with your details

Forgot password? Click here to reset