DeepAI AI Chat
Log In Sign Up

Multi-channel Transformers for Multi-articulatory Sign Language Translation

by   Necati Cihan Camgoz, et al.

Sign languages use multiple asynchronous information channels (articulators), not just the hands but also the face and body, which computational approaches often ignore. In this paper we tackle the multi-articulatory sign language translation task and propose a novel multi-channel transformer architecture. The proposed architecture allows both the inter and intra contextual relationships between different sign articulators to be modelled within the transformer network itself, while also maintaining channel specific information. We evaluate our approach on the RWTH-PHOENIX-Weather-2014T dataset and report competitive translation performance. Importantly, we overcome the reliance on gloss annotations which underpin other state-of-the-art approaches, thereby removing future need for expensive curated datasets.


page 1

page 2

page 3

page 4


Adversarial Training for Multi-Channel Sign Language Production

Sign Languages are rich multi-channel languages, requiring articulation ...

Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation

Prior work on Sign Language Translation has shown that having a mid-leve...

A Transformer-Based Contrastive Learning Approach for Few-Shot Sign Language Recognition

Sign language recognition from sequences of monocular images or 2D poses...

Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discourse

Sign Language Translation (SLT) is a challenging task that aims to gener...

Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives

It is common practice to represent spoken languages at their phonetic le...

SimulSLT: End-to-End Simultaneous Sign Language Translation

Sign language translation as a kind of technology with profound social s...

SignNet: Single Channel Sign Generation using Metric Embedded Learning

A true interpreting agent not only understands sign language and transla...