RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairs

by   Zhouxia Wang, et al.

Blind face restoration aims at recovering high-quality face images from those with unknown degradations. Current algorithms mainly introduce priors to complement high-quality details and achieve impressive progress. However, most of these algorithms ignore abundant contextual information in the face and its interplay with the priors, leading to sub-optimal performance. Moreover, they pay less attention to the gap between the synthetic and real-world scenarios, limiting the robustness and generalization to real-world applications. In this work, we propose RestoreFormer++, which on the one hand introduces fully-spatial attention mechanisms to model the contextual information and the interplay with the priors, and on the other hand, explores an extending degrading model to help generate more realistic degraded face images to alleviate the synthetic-to-real-world gap. Compared with current algorithms, RestoreFormer++ has several crucial benefits. First, instead of using a multi-head self-attention mechanism like the traditional visual transformer, we introduce multi-head cross-attention over multi-scale features to fully explore spatial interactions between corrupted information and high-quality priors. In this way, it can facilitate RestoreFormer++ to restore face images with higher realness and fidelity. Second, in contrast to the recognition-oriented dictionary, we learn a reconstruction-oriented dictionary as priors, which contains more diverse high-quality facial details and better accords with the restoration target. Third, we introduce an extending degrading model that contains more realistic degraded scenarios for training data synthesizing, and thus helps to enhance the robustness and generalization of our RestoreFormer++ model. Extensive experiments show that RestoreFormer++ outperforms state-of-the-art algorithms on both synthetic and real-world datasets.


page 2

page 5

page 6

page 8

page 10

page 12

page 13

page 15


RestoreFormer: High-Quality Blind Face Restoration From Undegraded Key-Value Pairs

Blind face restoration is to recover a high-quality face image from unkn...

Towards Real-World Blind Face Restoration with Generative Facial Prior

Blind face restoration usually relies on facial priors, such as facial g...

RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors

Existing dehazing approaches struggle to process real-world hazy images ...

Learning Dual Memory Dictionaries for Blind Face Restoration

To improve the performance of blind face restoration, recent works mainl...

HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment

Existing face restoration researches typically relies on either the degr...

Blind Face Restoration via Deep Multi-scale Component Dictionaries

Recent reference-based face restoration methods have received considerab...

Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior

Facial semantic guidance (facial landmarks, facial parsing maps, facial ...

Please sign up or login with your details

Forgot password? Click here to reset