ChatFace: Chat-Guided Real Face Editing via Diffusion Latent Space Manipulation

by   Dongxu Yue, et al.

Editing real facial images is a crucial task in computer vision with significant demand in various real-world applications. While GAN-based methods have showed potential in manipulating images especially when combined with CLIP, these methods are limited in their ability to reconstruct real images due to challenging GAN inversion capability. Despite the successful image reconstruction achieved by diffusion-based methods, there are still challenges in effectively manipulating fine-gained facial attributes with textual instructions.To address these issues and facilitate convenient manipulation of real facial images, we propose a novel approach that conduct text-driven image editing in the semantic latent space of diffusion model. By aligning the temporal feature of the diffusion model with the semantic condition at generative process, we introduce a stable manipulation strategy, which perform precise zero-shot manipulation effectively. Furthermore, we develop an interactive system named ChatFace, which combines the zero-shot reasoning ability of large language models to perform efficient manipulations in diffusion semantic latent space. This system enables users to perform complex multi-attribute manipulations through dialogue, opening up new possibilities for interactive image editing. Extensive experiments confirmed that our approach outperforms previous methods and enables precise editing of real facial images, making it a promising candidate for real-world applications. Project page:


page 8

page 16

page 18

page 19

page 20

page 21

page 23

page 24


Zero-shot Inversion Process for Image Attribute Editing with Diffusion Models

Denoising diffusion models have shown outstanding performance in image e...

Diffusion Models already have a Semantic Latent Space

Diffusion models achieve outstanding generative performance in various d...

Talk-to-Edit: Fine-Grained Facial Editing via Dialog

Facial editing is an important task in vision and graphics with numerous...

Zero-shot Text-driven Physically Interpretable Face Editing

This paper proposes a novel and physically interpretable method for face...

Local 3D Editing via 3D Distillation of CLIP Knowledge

3D content manipulation is an important computer vision task with many r...

Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation

Morphable models are essential for the statistical modeling of 3D faces....

FreeDrag: Point Tracking is Not What You Need for Interactive Point-based Image Editing

To serve the intricate and varied demands of image editing, precise and ...

Please sign up or login with your details

Forgot password? Click here to reset