PISE: Person Image Synthesis and Editing with Decoupled GAN

by   Jinsong Zhang, et al.

Person image synthesis, e.g., pose transfer, is a challenging problem due to large variation and occlusion. Existing methods have difficulties predicting reasonable invisible regions and fail to decouple the shape and style of clothing, which limits their applications on person image editing. In this paper, we propose PISE, a novel two-stage generative model for Person Image Synthesis and Editing, which is able to generate realistic person images with desired poses, textures, or semantic layouts. For human pose transfer, we first synthesize a human parsing map aligned with the target pose to represent the shape of clothing by a parsing generator, and then generate the final image by an image generator. To decouple the shape and style of clothing, we propose joint global and local per-region encoding and normalization to predict the reasonable style of clothing for invisible regions. We also propose spatial-aware normalization to retain the spatial context relationship in the source image. The results of qualitative and quantitative experiments demonstrate the superiority of our model on human pose transfer. Besides, the results of texture transfer and region editing show that our model can be applied to person image editing.


page 1

page 3

page 6

page 7

page 8


Learning Semantic Person Image Generation by Region-Adaptive Normalization

Human pose transfer has received great attention due to its wide applica...

Human Pose Transfer by Adaptive Hierarchical Deformation

Human pose transfer, as a misaligned image generation task, is very chal...

Style Your Hair: Latent Optimization for Pose-Invariant Hairstyle Transfer via Local-Style-Aware Hair Alignment

Editing hairstyle is unique and challenging due to the complexity and de...

GLocal: Global Graph Reasoning and Local Structure Transfer for Person Image Generation

In this paper, we focus on person image generation, namely, generating p...

Improving Human Image Synthesis with Residual Fast Fourier Transformation and Wasserstein Distance

With the rapid development of the Metaverse, virtual humans have emerged...

Combining Attention with Flow for Person Image Synthesis

Pose-guided person image synthesis aims to synthesize person images by t...

SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer

In recent years, virtual makeup applications have become more and more p...

Please sign up or login with your details

Forgot password? Click here to reset