research
∙
06/12/2023
Transformers learn through gradual rank increase
We identify incremental learning dynamics in transformers, where the dif...
share
research
∙
07/15/2022
Position Prediction as an Effective Pretraining Strategy
Transformers have gained increasing popularity in a wide range of applic...
research
∙
06/10/2022
The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon
The grokking phenomenon as reported by Power et al. ( arXiv:2201.02177 )...
research
∙
01/28/2022
Efficient Embedding of Semantic Similarity in Control Policies via Entangled Bisimulation
Learning generalizeable policies from visual input in the presence of vi...
research
∙
05/17/2021
Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning
Offline Reinforcement Learning promises to learn effective policies from...
research
∙
06/13/2020