Fingertip Detection and Tracking for Recognition of Air-Writing in Videos

by   Sohom Mukherjee, et al.

Air-writing is the process of writing characters or words in free space using finger or hand movements without the aid of any hand-held device. In this work, we address the problem of mid-air finger writing using web-cam video as input. In spite of recent advances in object detection and tracking, accurate and robust detection and tracking of the fingertip remains a challenging task, primarily due to small dimension of the fingertip. Moreover, the initialization and termination of mid-air finger writing is also challenging due to the absence of any standard delimiting criterion. To solve these problems, we propose a new writing hand pose detection algorithm for initialization of air-writing using the Faster R-CNN framework for accurate hand detection followed by hand segmentation and finally counting the number of raised fingers based on geometrical properties of the hand. Further, we propose a robust fingertip detection and tracking approach using a new signature function called distance-weighted curvature entropy. Finally, a fingertip velocity-based termination criterion is used as a delimiter to mark the completion of the air-writing gesture. Experiments show the superiority of the proposed fingertip detection and tracking algorithm over state-of-the-art approaches giving a mean precision of 73.1 condition which is of vital importance to air-writing. Character recognition experiments give a mean accuracy of 96.11 system, a result which is comparable to that of existing handwritten character recognition systems.


page 10

page 12

page 15


Spectral Graph-based Features for Recognition of Handwritten Characters: A Case Study on Handwritten Devanagari Numerals

Interpretation of different writing styles, unconstrained cursiveness an...

AirPen: A Touchless Fingertip Based Gestural Interface for Smartphones and Head-Mounted Devices

Hand gestures are an intuitive, socially acceptable, and a non-intrusive...

Hand Tracking based on Hierarchical Clustering of Range Data

Fast and robust hand segmentation and tracking is an essential basis for...

Robust and customized methods for real-time hand gesture recognition under object-occlusion

Dynamic hand tracking and gesture recognition is a hard task since there...

RNN-based Online Handwritten Character Recognition Using Accelerometer and Gyroscope Data

This abstract explores an RNN-based approach to online handwritten recog...

Air-Writing Translater: A Novel Unsupervised Domain Adaptation Method for Inertia-Trajectory Translation of In-air Handwriting

As a new way of human-computer interaction, inertial sensor based in-air...

UV Direct-Writing of Metals on Polyimide

Conductive micro-patterned copper tracks were fabricated by UV direct-wr...

Please sign up or login with your details

Forgot password? Click here to reset