End-to-End Eye Movement Detection Using Convolutional Neural Networks

by   Sabrina Hoppe, et al.

Common computational methods for automated eye movement detection - i.e. the task of detecting different types of eye movement in a continuous stream of gaze data - are limited in that they either involve thresholding on hand-crafted signal features, require individual detectors each only detecting a single movement, or require pre-segmented data. We propose a novel approach for eye movement detection that only involves learning a single detector end-to-end, i.e. directly from the continuous gaze data stream and simultaneously for different eye movements without any manual feature crafting or segmentation. Our method is based on convolutional neural networks (CNN) that recently demonstrated superior performance in a variety of tasks in computer vision, signal processing, and machine learning. We further introduce a novel multi-participant dataset that contains scripted and free-viewing sequences of ground-truth annotated saccades, fixations, and smooth pursuits. We show that our CNN-based method outperforms state-of-the-art baselines by a large margin on this challenging dataset, thereby underlining the significant potential of this approach for holistic, robust, and accurate eye movement protocol analysis.


page 6

page 8


Hierarchical HMM for Eye Movement Classification

In this work, we tackle the problem of ternary eye movement classificati...

MagicEyes: A Large Scale Eye Gaze Estimation Dataset for Mixed Reality

With the emergence of Virtual and Mixed Reality (XR) devices, eye tracki...

A Two-stream End-to-End Deep Learning Network for Recognizing Atypical Visual Attention in Autism Spectrum Disorder

Eye movements have been widely investigated to study the atypical visual...

Deep semantic gaze embedding and scanpath comparison for expertise classification during OPT viewing

Modeling eye movement indicative of expertise behavior is decisive in us...

FROG: A new people detection dataset for knee-high 2D range finders

Mobile robots require knowledge of the environment, especially of humans...

Assessment of Shift-Invariant CNN Gaze Mappings for PS-OG Eye Movement Sensors

Photosensor oculography (PS-OG) eye movement sensors offer desirable per...

Please sign up or login with your details

Forgot password? Click here to reset