Are you really looking at me? A Framework for Extracting Interpersonal Eye Gaze from Conventional Video

by   Minh Tran, et al.

Despite a revolution in the pervasiveness of video cameras in our daily lives, one of the most meaningful forms of nonverbal affective communication, interpersonal eye gaze, i.e. eye gaze relative to a conversation partner, is not available from common video. We introduce the Interpersonal-Calibrating Eye-gaze Encoder (ICE), which automatically extracts interpersonal gaze from video recordings without specialized hardware and without prior knowledge of participant locations. Leveraging the intuition that individuals spend a large portion of a conversation looking at each other enables the ICE dynamic clustering algorithm to extract interpersonal gaze. We validate ICE in both video chat using an objective metric with an infrared gaze tracker (F1=0.846, N=8), as well as in face-to-face communication with expert-rated evaluations of eye contact (r= 0.37, N=170). We then use ICE to analyze behavior in two different, yet important affective communication domains: interrogation-based deception detection, and communication skill assessment in speed dating. We find that honest witnesses break interpersonal gaze contact and look down more often than deceptive witnesses when answering questions (p=0.004, d=0.79). In predicting expert communication skill ratings in speed dating videos, we demonstrate that interpersonal gaze alone has more predictive power than facial expressions.


page 3

page 8

page 11

page 14


Gaze-based Autism Detection for Adolescents and Young Adults using Prosaic Videos

Autism often remains undiagnosed in adolescents and adults. Prior resear...

Learning Video-independent Eye Contact Segmentation from In-the-Wild Videos

Human eye contact is a form of non-verbal communication and can have a g...

LookAtChat: Visualizing Gaze Awareness for Remote Small-Group Conversations

Video conferences play a vital role in our daily lives. However, many no...

Custom Video-Oculography Device and Its Application to Fourth Purkinje Image Detection during Saccades

We built a custom video-based eye-tracker that saves every video frame a...

Restoring Eye Contact to the Virtual Classroom with Machine Learning

Nonverbal communication, in particular eye contact, is a critical elemen...

The Quiet Eye Phenomenon in Minimally Invasive Surgery

In this paper, we report our discovery of a gaze behavior called Quiet E...

Eye Contact Correction using Deep Neural Networks

In a typical video conferencing setup, it is hard to maintain eye contac...

Please sign up or login with your details

Forgot password? Click here to reset