research
∙
08/30/2023
LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models
In recent years, there have been remarkable advancements in the performa...
share
research
∙
05/22/2023
Learning Easily Updated General Purpose Text Representations with Adaptable Task-Specific Prefixes
Many real-world applications require making multiple predictions from th...
research
∙
04/04/2023
Defending Against Patch-based Backdoor Attacks on Self-Supervised Learning
Recently, self-supervised learning (SSL) was shown to be vulnerable to p...
research
∙
02/04/2023
Representation Deficiency in Masked Language Modeling
Masked Language Modeling (MLM) has been one of the most prominent approa...
research
∙
11/04/2022
Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler
Active learning with strong and weak labelers considers a practical sett...
research
∙
06/02/2022
BayesFormer: Transformer with Uncertainty Estimation
Transformer has become ubiquitous due to its dominant performance in var...
research
∙
04/12/2022
Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem
We propose an autoregressive entity linking model, that is trained with ...
research
∙
04/09/2022
IDPG: An Instance-Dependent Prompt Generation Method
Prompt tuning is a new, efficient NLP transfer learning paradigm that ad...
research
∙
12/07/2021
Reducing Target Group Bias in Hate Speech Detectors
The ubiquity of offensive and hateful content on online fora necessitate...
research
∙
10/16/2021
Sparse Distillation: Speeding Up Text Classification by Using Bigger Models
Distilling state-of-the-art transformer models into lightweight student ...
research
∙
06/03/2021
Luna: Linear Unified Nested Attention
The quadratic computational and memory complexities of the Transformer's...
research
∙
04/29/2021
Entailment as Few-Shot Learner
Large pre-trained language models (LMs) have demonstrated remarkable abi...
research
∙
04/18/2021
On the Influence of Masking Policies in Intermediate Pre-training
Current NLP models are predominantly trained through a pretrain-then-fin...
research
∙
04/12/2021
On Unifying Misinformation Detection
In this paper, we introduce UnifiedM2, a general-purpose misinformation ...
research
∙
12/31/2020
Studying Strategically: Learning to Mask for Closed-book QA
Closed-book question-answering (QA) is a challenging task that requires ...
research
∙
12/31/2020
CLEAR: Contrastive Learning for Sentence Representation
Pre-trained language models have proven their unique powers in capturing...
research
∙
06/15/2020
To Pretrain or Not to Pretrain: Examining the Benefits of Pretraining on Resource Rich Tasks
Pretraining NLP models with variants of Masked Language Model (MLM) obje...
research
∙
06/08/2020
Linformer: Self-Attention with Linear Complexity
Large transformer models have shown extraordinary success in achieving s...
research
∙
06/07/2020
Language Models as Fact Checkers?
Recent work has suggested that language models (LMs) store both common-s...
research
∙
11/07/2019
Blockwise Self-Attention for Long Document Understanding
We present BlockBERT, a lightweight and efficient BERT model that is des...
research
∙
01/23/2019
Fundamental Limits of Approximate Gradient Coding
It has been established that when the gradient coding problem is distrib...
research
∙
04/25/2018
Fundamental Limits of Coded Linear Transform
In large scale distributed linear transform problems, coded computation ...
research
∙
04/16/2018
UCBoost: A Boosting Approach to Tame Complexity and Optimality for Stochastic Bandits
In this work, we address the open problem of finding low-complexity near...
research
∙
02/09/2018