SPColor: Semantic Prior Guided Exemplar-based Image Colorization

by   Siqi Chen, et al.
Beijing University of Posts and Telecommunications
NetEase, Inc

Exemplar-based image colorization aims to colorize a target grayscale image based on a color reference image, and the key is to establish accurate pixel-level semantic correspondence between these two images. Previous methods search for correspondence across the entire reference image, and this type of global matching is easy to get mismatch. We summarize the difficulties in two aspects: (1) When the reference image only contains a part of objects related to target image, improper correspondence will be established in unrelated regions. (2) It is prone to get mismatch in regions where the shape or texture of the object is easily confused. To overcome these issues, we propose SPColor, a semantic prior guided exemplar-based image colorization framework. Different from previous methods, SPColor first coarsely classifies pixels of the reference and target images to several pseudo-classes under the guidance of semantic prior, then the correspondences are only established locally between the pixels in the same class via the newly designed semantic prior guided correspondence network. In this way, improper correspondence between different semantic classes is explicitly excluded, and the mismatch is obviously alleviated. Besides, to better reserve the color from reference, a similarity masked perceptual loss is designed. Noting that the carefully designed SPColor utilizes the semantic prior provided by an unsupervised segmentation model, which is free for additional manual semantic annotations. Experiments demonstrate that our model outperforms recent state-of-the-art methods both quantitatively and qualitatively on public dataset.


page 1

page 3

page 6

page 7

page 8

page 9


Reference-Based Video Colorization with Spatiotemporal Correspondence

We propose a novel reference-based video colorization framework with spa...

Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization

Exemplar-based colorization approaches rely on reference image to provid...

Temporal Consistent Automatic Video Colorization via Semantic Correspondence

Video colorization task has recently attracted wide attention. Recent me...

Cross-domain Correspondence Learning for Exemplar-based Image Translation

We present a general framework for exemplar-based image translation, whi...

Examplar-Based Face Colorization Using Image Morphing

Colorization of gray-scale images relies on prior color information. Exa...

TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations

Image inpainting is the task of plausibly restoring missing pixels withi...

Ref-NPR: Reference-Based Non-Photorealistic Radiance Fields

Existing 3D scene stylization methods employ an arbitrary style referenc...

Please sign up or login with your details

Forgot password? Click here to reset