MINTIME: Multi-Identity Size-Invariant Video Deepfake Detection

In this paper, we introduce MINTIME, a video deepfake detection approach that captures spatial and temporal anomalies and handles instances of multiple people in the same video and variations in face sizes. Previous approaches disregard such information either by using simple a-posteriori aggregation schemes, i.e., average or max operation, or using only one identity for the inference, i.e., the largest one. On the contrary, the proposed approach builds on a Spatio-Temporal TimeSformer combined with a Convolutional Neural Network backbone to capture spatio-temporal anomalies from the face sequences of multiple identities depicted in a video. This is achieved through an Identity-aware Attention mechanism that attends to each face sequence independently based on a masking operation and facilitates video-level aggregation. In addition, two novel embeddings are employed: (i) the Temporal Coherent Positional Embedding that encodes each face sequence's temporal information and (ii) the Size Embedding that encodes the size of the faces as a ratio to the video frame size. These extensions allow our system to adapt particularly well in the wild by learning how to aggregate information of multiple identities, which is usually disregarded by other methods in the literature. It achieves state-of-the-art results on the ForgeryNet dataset with an improvement of up to 14 demonstrates ample generalization capabilities in cross-forgery and cross-dataset settings. The code is publicly available at


page 4

page 5

page 8


Self-attention aggregation network for video face representation and recognition

Models based on self-attention mechanisms have been successful in analyz...

Explore Spatio-temporal Aggregation for Insubstantial Object Detection: Benchmark Dataset and Baseline

We endeavor on a rarely explored task named Insubstantial Object Detecti...

Spatio-temporal Co-attention Fusion Network for Video Splicing Localization

Digital video splicing has become easy and ubiquitous. Malicious users c...

Video Shadow Detection via Spatio-Temporal Interpolation Consistency Training

It is challenging to annotate large-scale datasets for supervised video ...

A Spatial-Temporal Deformable Attention based Framework for Breast Lesion Detection in Videos

Detecting breast lesion in videos is crucial for computer-aided diagnosi...

Identity-aware Graph Memory Network for Action Detection

Action detection plays an important role in high-level video understandi...

A Spatio-Temporal Identity Verification Method for Person-Action Instance Search in Movies

As one of the challenging problems in video search, Person-Action Instan...

Please sign up or login with your details

Forgot password? Click here to reset