DeepAI AI Chat
Log In Sign Up

MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language

by   Hamid Reza Vaezi Joze, et al.

Computer Vision has been improved significantly in the past few decades. It has enabled machine to do many human tasks. However, the real challenge is in enabling machine to carry out tasks that an average human does not have the skills for. One such challenge that we have tackled in this paper is providing accessibility for deaf individual by providing means of communication with others with the aid of computer vision. Unlike other frequent works focusing on multiple camera, depth camera, electrical glove or visual gloves, we focused on the sole use of RGB which allows everybody to communicate with a deaf individual through their personal devices. This is not a new approach but the lack of realistic large-scale data set prevented recent computer vision trends on video classification in this filed. In this paper, we propose the first large scale ASL data set that covers over 200 signers, signer independent sets, challenging and unconstrained recording conditions and a large class count of 1000 signs. We evaluate baselines from action recognition techniques on the data set. We propose I3D, known from video classifications, as a powerful and suitable architecture for sign language recognition. We also propose new pre-trained model more appropriate for sign language recognition. Finally, We estimate the effect of number of classes and number of training samples on the recognition accuracy.


page 6

page 8


Skeleton Based Sign Language Recognition Using Whole-body Keypoints

Sign language is a visual language that is used by deaf or speech impair...

BosphorusSign22k Sign Language Recognition Dataset

Sign Language Recognition is a challenging research domain. It has recen...

NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding

Research on depth-based human activity analysis achieved outstanding per...

A Comprehensive Study on Sign Language Recognition Methods

In this paper, a comparative experimental assessment of computer vision-...

DIY Human Action Data Set Generation

The recent successes in applying deep learning techniques to solve stand...

Sign Language Recognition Analysis using Multimodal Data

Voice-controlled personal and home assistants (such as the Amazon Echo a...

Fingerspelling recognition in the wild with iterative visual attention

Sign language recognition is a challenging gesture sequence recognition ...