Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classification

by   Zhenhailong Wang, et al.
University of Illinois at Urbana-Champaign

State-of-the-art brain-to-text systems have achieved great success in decoding language directly from brain signals using neural networks. However, current approaches are limited to small closed vocabularies which are far from enough for natural communication. In addition, most of the high-performing approaches require data from invasive devices (e.g., ECoG). In this paper, we extend the problem to open vocabulary Electroencephalography(EEG)-To-Text Sequence-To-Sequence decoding and zero-shot sentence sentiment classification on natural reading tasks. We hypothesis that the human brain functions as a special text encoder and propose a novel framework leveraging pre-trained language models (e.g., BART). Our model achieves a 40.1 EEG-To-Text decoding and a 55.6 sentiment classification, which significantly outperforms supervised baselines. Furthermore, we show that our proposed model can handle data from various subjects and sources, showing great potential for a high-performance open vocabulary brain-to-text system once sufficient data is available


page 1

page 2

page 3

page 4


Zero-Shot Text Classification via Self-Supervised Tuning

Existing solutions to zero-shot text classification either conduct promp...

Decoding EEG Brain Activity for Multi-Modal Natural Language Processing

Until recently, human behavioral data from reading has mainly been of in...

UniCoRN: Unified Cognitive Signal ReconstructioN bridging cognitive signals and human language

Decoding text stimuli from cognitive signals (e.g. fMRI) enhances our un...

Brain2Char: A Deep Architecture for Decoding Text from Brain Recordings

Decoding language representations directly from the brain can enable new...

SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders

We propose an end-to-end neural model for zero-shot abstractive text sum...

Cross-view Brain Decoding

How the brain captures the meaning of linguistic stimuli across multiple...

Open Aspect Target Sentiment Classification with Natural Language Prompts

For many business applications, we often seek to analyze sentiments asso...

Please sign up or login with your details

Forgot password? Click here to reset