research
∙
06/23/2023
Scaling MLPs: A Tale of Inductive Bias
In this work we revisit the most fundamental building block in deep lear...
share
research
∙
05/25/2023
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers
Autoregressive Transformers adopted in Large Language Models (LLMs) are ...
research
∙
04/14/2023
OpenAssistant Conversations – Democratizing Large Language Model Alignment
Aligning large language models (LLMs) with human preferences has proven ...
research
∙
02/23/2023
Random Teachers are Good Teachers
In this work, we investigate the implicit regularization induced by teac...
research
∙
11/22/2022
Cosmology from Galaxy Redshift Surveys with PointNet
In recent years, deep learning approaches have achieved state-of-the-art...
research
∙
10/25/2022
The Curious Case of Benign Memorization
Despite the empirical advances of deep learning across a variety of lear...
research
∙
10/03/2022
Mastering Spatial Graph Prediction of Road Networks
Accurately predicting road networks from satellite images requires a glo...
research
∙
06/07/2022
Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse
Transformers have achieved remarkable success in several domains, rangin...
research
∙
02/22/2021