Collaging on Internal Representations: An Intuitive Approach for Semantic Transfiguration

by   Ryohei Suzuki, et al.

We present a novel CNN-based image editing method that allows the user to change the semantic information of an image over a user-specified region. Our method makes this possible by combining the idea of manifold projection with spatial conditional batch normalization (sCBN), a version of conditional batch normalization with user-specifiable spatial weight maps. With sCBN and manifold projection, our method lets the user perform (1) spatial class translation that changes the class of an object over an arbitrary region of user's choice, and (2) semantic transplantation that transplants semantic information contained in an arbitrary region of the reference image to an arbitrary region in the target image. These two transformations can be used simultaneously, and can realize a complex composite image-editing task like "change the nose of a beagle to that of a bulldog, and open her mouth." The user can also use our method with intuitive copy-paste-style manipulations. We demonstrate the power of our method on various images. Code will be available at


page 2

page 5

page 8

page 9

page 10

page 11

page 15

page 16


Improving the Harmony of the Composite Image by Spatial-Separated Attention Module

Image composition is one of the most important applications in image pro...

Semantic Image Synthesis with Spatially-Adaptive Normalization

We propose spatially-adaptive normalization, a simple but effective laye...

SEAN: Image Synthesis with Semantic Region-Adaptive Normalization

We propose semantic region-adaptive normalization (SEAN), a simple but e...

Instance Normalization: The Missing Ingredient for Fast Stylization

It this paper we revisit the fast stylization method introduced in Ulyan...

Differential Diffusion: Giving Each Pixel Its Strength

Text-based image editing has advanced significantly in recent years. Wit...

Rethinking Spatially-Adaptive Normalization

Spatially-adaptive normalization is remarkably successful recently in co...

Please sign up or login with your details

Forgot password? Click here to reset