Multi-spectral Class Center Network for Face Manipulation Detection and Localization

by   Changtao Miao, et al.

As Deepfake contents continue to proliferate on the internet, advancing face manipulation forensics has become a pressing issue. To combat this emerging threat, previous methods mainly focus on studying how to distinguish authentic and manipulated face images. Despite impressive, image-level classification lacks explainability and is limited to some specific application scenarios. Existing forgery localization methods suffer from imprecise and inconsistent pixel-level annotations. To alleviate these problems, this paper first re-constructs the FaceForensics++ dataset by introducing pixel-level annotations, then builds an extensive benchmark for localizing tampered regions. Next, a novel Multi-Spectral Class Center Network (MSCCNet) is proposed for face manipulation detection and localization. Specifically, inspired by the power of frequency-related forgery traces, we design Multi-Spectral Class Center (MSCC) module to learn more generalizable and semantic-agnostic features. Based on the features of different frequency bands, the MSCC module collects multispectral class centers and computes pixel-to-class relations. Applying multi-spectral class-level representations suppresses the semantic information of the visual concepts, which is insensitive to manipulations. Furthermore, we propose a Multi-level Features Aggregation (MFA) module to employ more low-level forgery artifacts and structure textures. Experimental results quantitatively and qualitatively indicate the effectiveness and superiority of the proposed MSCCNet on comprehensive localization benchmarks. We expect this work to inspire more studies on pixel-level face manipulation localization. The annotations and code will be available.


page 1

page 3

page 8


Global Weighted Average Pooling Bridges Pixel-level Localization and Image-level Classification

In this work, we first tackle the problem of simultaneous pixel-level lo...

Zooming into Face Forensics: A Pixel-level Analysis

The stunning progress in face manipulation methods has made it possible ...

Inter-Image Communication for Weakly Supervised Localization

Weakly supervised localization aims at finding target object regions usi...

Learning Hierarchical Semantic Image Manipulation through Structured Representations

Understanding, reasoning, and manipulating semantic concepts of images h...

MTU-Net: Multi-level TransUNet for Space-based Infrared Tiny Ship Detection

Space-based infrared tiny ship detection aims at separating tiny ships f...

Pixel Adaptive Deep Unfolding Transformer for Hyperspectral Image Reconstruction

Hyperspectral Image (HSI) reconstruction has made gratifying progress wi...

Rethinking Gradient Operator for Exposing AI-enabled Face Forgeries

For image forensics, convolutional neural networks (CNNs) tend to learn ...

Please sign up or login with your details

Forgot password? Click here to reset