GAVIN: Gaze-Assisted Voice-Based Implicit Note-taking

by   Anam Ahmad Khan, et al.

Annotation is an effective reading strategy people often undertake while interacting with digital text. It involves highlighting pieces of text and making notes about them. Annotating while reading in a desktop environment is considered trivial but, in a mobile setting where people read while hand-holding devices, the task of highlighting and typing notes on a mobile display is challenging. In this paper, we introduce GAVIN, a gaze-assisted voice note-taking application, which enables readers to seamlessly take voice notes on digital documents by implicitly anchoring them to text passages. We first conducted a contextual enquiry focusing on participants' note-taking practices on digital documents. Using these findings, we propose a method which leverages eye-tracking and machine learning techniques to annotate voice notes with reference text passages. To evaluate our approach, we recruited 32 participants performing voice note-taking. Following, we trained a classifier on the data collected to predict text passage where participants made voice notes. Lastly, we employed the classifier to built GAVIN and conducted a user study to demonstrate the feasibility of the system. This research demonstrates the feasibility of using gaze as a resource for implicit anchoring of voice notes, enabling the design of systems that allow users to record voice notes with minimal effort and high accuracy.


page 13

page 15

page 26

page 27

page 28

page 29

page 30

page 31


Using voice note-taking to promote learners' conceptual understanding

Though recent technological advances have enabled note-taking through di...

The Walking Talking Stick: Understanding Automated Note-Taking in Walking Meetings

While walking meetings offer a healthy alternative to sit-down meetings,...

iVRNote: Design, Creation and Evaluation of an Interactive Note-Taking Interface for Study and Reflection in VR Learning Environments

In this contribution, we design, implement and evaluate the pedagogical ...

Gaze-Driven Adaptive Interventions for Magazine-Style Narrative Visualizations

In this paper we investigate the value of gaze-driven adaptive intervent...

A preliminary study on evaluating Consultation Notes with Post-Editing

Automatic summarisation has the potential to aid physicians in streamlin...

Creation and Detection of German Voice Deepfakes

Synthesizing voice with the help of machine learning techniques has made...

Hear Me Out: A Study on the Use of the Voice Modality for Crowdsourced Relevance Assessments

The creation of relevance assessments by human assessors (often nowadays...

Please sign up or login with your details

Forgot password? Click here to reset