Temporal Consistent Automatic Video Colorization via Semantic Correspondence

by   Yu Zhang, et al.
Beijing University of Posts and Telecommunications
NetEase, Inc

Video colorization task has recently attracted wide attention. Recent methods mainly work on the temporal consistency in adjacent frames or frames with small interval. However, it still faces severe challenge of the inconsistency between frames with large interval.To address this issue, we propose a novel video colorization framework, which combines semantic correspondence into automatic video colorization to keep long-range consistency. Firstly, a reference colorization network is designed to automatically colorize the first frame of each video, obtaining a reference image to supervise the following whole colorization process. Such automatically colorized reference image can not only avoid labor-intensive and time-consuming manual selection, but also enhance the similarity between reference and grayscale images. Afterwards, a semantic correspondence network and an image colorization network are introduced to colorize a series of the remaining frames with the help of the reference. Each frame is supervised by both the reference image and the immediately colorized preceding frame to improve both short-range and long-range temporal consistency. Extensive experiments demonstrate that our method outperforms other methods in maintaining temporal consistency both qualitatively and quantitatively. In the NTIRE 2023 Video Colorization Challenge, our method ranks at the 3rd place in Color Distribution Consistency (CDC) Optimization track.


page 2

page 3

page 4

page 5

page 6

page 7

page 8


Deep Exemplar-based Video Colorization

This paper presents the first end-to-end network for exemplar-based vide...

Reference-Based Video Colorization with Spatiotemporal Correspondence

We propose a novel reference-based video colorization framework with spa...

Attention-Aware Anime Line Drawing Colorization

Automatic colorization of anime line drawing has attracted much attentio...

SPColor: Semantic Prior Guided Exemplar-based Image Colorization

Exemplar-based image colorization aims to colorize a target grayscale im...

Retrieval and Registration of Long-Range Overlapping Frames for Scalable Mosaicking of In Vivo Fetoscopy

Purpose: The standard clinical treatment of Twin-to-Twin Transfusion Syn...

Temporal Tessellation: A Unified Approach for Video Analysis

We present a general approach to video understanding, inspired by semant...

Self-Supervised Correspondence Estimation via Multiview Registration

Video provides us with the spatio-temporal consistency needed for visual...

Please sign up or login with your details

Forgot password? Click here to reset