Pose Randomization for Weakly Paired Image Style Translation

by   Zexi Chen, et al.

Utilizing the trained model under different conditions without data annotation is attractive for robot applications. Towards this goal, one class of methods is to translate the image style from the training environment to the current one. Conventional studies on image style translation mainly focus on two settings: paired data on images from two domains with exactly aligned content, and unpaired data, with independent content. In this paper, we would like to propose a new setting, where the content in the two images is aligned with error in poses. We consider that this setting is more practical since robots with various sensors are able to align the data up to some error level, even with different styles. To solve this problem, we propose PRoGAN to learn a style translator by intentionally transforming the original domain images with a noisy pose, then matching the distribution of translated transformed images and the distribution of the target domain images. The adversarial training enforces the network to learn the style translation, avoiding being entangled with other variations. In addition, we propose two pose estimation based self-supervised tasks to further improve the performance. Finally, PRoGAN is validated on both simulated and real-world collected data to show the effectiveness. Results on down-stream tasks, classification, road segmentation, object detection, and feature matching show its potential for real applications. https://github.com/wrld/PRoGAN .


page 1

page 5

page 6

page 7

page 8


Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

Image-to-image translation is a class of vision and graphics problems wh...

Level-aware Haze Image Synthesis by Self-Supervised Content-Style Disentanglement

The key procedure of haze image translation through adversarial training...

Analogical Image Translation for Fog Generation

Image-to-image translation is to map images from a given style to anothe...

Panoptic-based Object Style-Align for Image-to-Image Translation

Despite remarkable recent progress in image translation, the complex sce...

Learn to Differ: Sim2Real Small Defection Segmentation Network

Recent studies on deep-learning-based small defection segmentation appro...

Distribution Matching Losses Can Hallucinate Features in Medical Image Translation

This paper discusses how distribution matching losses, such as those use...

Rethinking Content and Style: Exploring Bias for Unsupervised Disentanglement

Content and style (C-S) disentanglement intends to decompose the underly...

Please sign up or login with your details

Forgot password? Click here to reset