research
∙
04/07/2021
Differentiable Patch Selection for Image Recognition
Neural Networks require large amounts of memory and compute to process h...
share
research
∙
03/05/2021
Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth
Attention-based architectures have become ubiquitous in machine learning...
research
∙
10/02/2020
Group Equivariant Stand-Alone Self-Attention For Vision
We provide a general self-attention formulation to impose group equivari...
research
∙
06/29/2020
Multi-Head Attention: Collaborate Instead of Concatenate
Attention layers are widely used in natural language processing (NLP) an...
research
∙
12/28/2019
Robust Cross-lingual Embeddings from Parallel Sentences
Recent advances in cross-lingual word embeddings have primarily relied o...
research
∙
11/08/2019
On the Relationship between Self-Attention and Convolutional Layers
Recent trends of incorporating attention mechanisms in vision have led r...
research
∙
03/18/2019
Extrapolating paths with graph neural networks
We consider the problem of path inference: given a path prefix, i.e., a ...
research
∙
09/20/2018