Gotcha: A Challenge-Response System for Real-Time Deepfake Detection

by   Govind Mittal, et al.

The integrity of online video interactions is threatened by the widespread rise of AI-enabled high-quality deepfakes that are now deployable in real-time. This paper presents Gotcha, a real-time deepfake detection system for live video interactions. The core principle underlying Gotcha is the presentation of a specially chosen cascade of both active and passive challenges to video conference participants. Active challenges include inducing changes in face occlusion, face expression, view angle, and ambiance; passive challenges include digital manipulation of the webcam feed. The challenges are designed to target vulnerabilities in the structure of modern deepfake generators and create perceptible artifacts for the human eye while inducing robust signals for ML-based automatic deepfake detectors. We present a comprehensive taxonomy of a large set of challenge tasks, which reveals a natural hierarchy among different challenges. Our system leverages this hierarchy by cascading progressively more demanding challenges to a suspected deepfake. We evaluate our system on a novel dataset of live users emulating deepfakes and show that our system provides consistent, measurable degradation of deepfake quality, showcasing its promise for robust real-time deepfake detection when deployed in the wild.


page 2

page 4

page 5

page 6

page 8

page 10

page 12

page 19


RealityTalk: Real-Time Speech-Driven Augmented Presentation for AR Live Storytelling

We present RealityTalk, a system that augments real-time live presentati...

PupilNet v2.0: Convolutional Neural Networks for CPU based real time Robust Pupil Detection

Real-time, accurate, and robust pupil detection is an essential prerequi...

Detection of Real-time DeepFakes in Video Conferencing with Active Probing and Corneal Reflection

The COVID pandemic has led to the wide adoption of online video calls in...

Real-time Multi-person Eyeblink Detection in the Wild for Untrimmed Video

Real-time eyeblink detection in the wild can widely serve for fatigue de...

Muscle Vision: Real Time Keypoint Based Pose Classification of Physical Exercises

Recent advances in machine learning technology have enabled highly porta...

A Survey On Video Forgery Detection

The Digital Forgeries though not visibly identifiable to human perceptio...

MeetScript: Designing Transcript-based Interactions to Support Active Participation in Group Video Meetings

While videoconferencing is prevalent, concurrent participation channels ...

Please sign up or login with your details

Forgot password? Click here to reset