DeepAI AI Chat
Log In Sign Up

Hand-Priming in Object Localization for Assistive Egocentric Vision

by   Kyungjun Lee, et al.
University of Maryland

Egocentric vision holds great promises for increasing access to visual information and improving the quality of life for people with visual impairments, with object recognition being one of the daily challenges for this population. While we strive to improve recognition performance, it remains difficult to identify which object is of interest to the user; the object may not even be included in the frame due to challenges in camera aiming without visual feedback. Also, gaze information, commonly used to infer the area of interest in egocentric vision, is often not dependable. However, blind users often tend to include their hand either interacting with the object that they wish to recognize or simply placing it in proximity for better camera aiming. We propose localization models that leverage the presence of the hand as the contextual information for priming the center area of the object of interest. In our approach, hand segmentation is fed to either the entire localization network or its last convolutional layers. Using egocentric datasets from sighted and blind individuals, we show that the hand-priming achieves higher precision than other approaches, such as fine-tuning, multi-class, and multi-task learning, which also encode hand-object interactions in localization.


page 1

page 4

page 5

page 6

page 7


Object Localization Assistive System Based on CV and Vibrotactile Encoding

Intelligent assistive systems can navigate blind people, but most of the...

Simultaneous prediction of hand gestures, handedness, and hand keypoints using thermal images

Hand gesture detection is a well-explored area in computer vision with a...

A Convolutional Neural Network based Live Object Recognition System as Blind Aid

This paper introduces a live object recognition system that serves as a ...

Using Web Co-occurrence Statistics for Improving Image Categorization

Object recognition and localization are important tasks in computer visi...

Analysis of Hand Segmentation in the Wild

A large number of works in egocentric vision have concentrated on action...

Foveated Haptic Gaze

As digital worlds become ubiquitous via video games, simulations, virtua...