Line Art Correlation Matching Network for Automatic Animation Colorization

by   Zhang Qian, et al.

Automatic animation line art colorization is a challenging computer vision problem, since the information of the line art is highly sparse and abstracted and there exists a strict requirement for the color and style consistency between frames. Recently, a lot of Generative Adversarial Network (GAN) based image-to-image translation methods for single line art colorization have emerged. They can generate perceptually appealing results conditioned on line art images. However, these methods can not be adopted for the purpose of animation colorization because there is a lack of consideration of the in-between frame consistency. Existing methods simply input the previous colored frame as a reference to color the next line art, which will mislead the colorization due to the spatial misalignment of the previous colored frame and the next line art especially at positions where apparent changes happen. To address these challenges, we design a kind of correlation matching model (called CM) to align the colored reference in a learnable way and integrate the model into an U-Net structure generator in a coarse-to-fine manner. This enables the generator to reconstruct the layer-wise synchronized features from the deep semantic code to the content progressively. Extension evaluations show that CM model can effectively improve the in-between consistency and the quality of colored frames especially when the motion is intense and diverse.


page 1

page 4

page 5

page 6

page 7


Unified Generative Adversarial Networks for Controllable Image-to-Image Translation

Controllable image-to-image translation, i.e., transferring an image fro...

User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks

Scribble colors based line art colorization is a challenging computer vi...

Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss

Line art colorization is expensive and challenging to automate. A GAN ap...

Automatic Temporally Coherent Video Colorization

Greyscale image colorization for applications in image restoration has s...

Artist-Guided Semiautomatic Animation Colorization

There is a delicate balance between automating repetitive work in creati...

TAI-GAN: Temporally and Anatomically Informed GAN for early-to-late frame conversion in dynamic cardiac PET motion correction

The rapid tracer kinetics of rubidium-82 (^82Rb) and high variation of c...

Deep Line Art Video Colorization with a Few References

Coloring line art images based on the colors of reference images is an i...

Please sign up or login with your details

Forgot password? Click here to reset