research
∙
07/03/2023
Trainable Transformer in Transformer
Recent works attribute the capability of in-context learning (ICL) in la...
share
research
∙
03/14/2023
Do Transformers Parse while Predicting the Masked Word?
Pre-trained language models have been shown to encode linguistic structu...
research
∙
02/13/2023
Task-Specific Skill Localization in Fine-tuned Language Models
Pre-trained language models can be fine-tuned to solve diverse NLP tasks...
research
∙
05/20/2022
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms
Approximating Stochastic Gradient Descent (SGD) as a Stochastic Differen...
research
∙
05/19/2022
Understanding Gradient Descent on Edge of Stability in Deep Learning
Deep learning experiments in Cohen et al. (2021) using deterministic Gra...
research
∙
05/31/2021
Learning and Generalization in RNNs
Simple recurrent neural networks (RNNs) and their more advanced cousins ...
research
∙
10/21/2019
Non-Gaussianity of Stochastic Gradient Noise
What enables Stochastic Gradient Descent (SGD) to achieve better general...
research
∙
08/16/2019
Effect of Activation Functions on the Training of Overparametrized Neural Nets
It is well-known that overparametrized neural networks trained using gra...
research
∙
03/10/2019
DeepTagRec: A Content-cum-User based Tag Recommendation Framework for Stack Overflow
In this paper, we develop a content-cum-user based deep learning framewo...
research
∙
12/02/2018
