research
∙
09/11/2023
Uncovering mesa-optimization algorithms in Transformers
Transformers have become the dominant model in deep learning, but the re...
share
research
∙
09/04/2023
Gated recurrent neural networks discover attention
Recent architectural developments have enabled recurrent neural networks...
research
∙
05/25/2023
Online learning of long-range dependencies
Online learning holds the promise of enabling efficient long-term credit...
research
∙
12/15/2022
Transformers learn in-context by gradient descent
Transformers have become the state-of-the-art neural network architectur...
research
∙
07/04/2022
The least-control principle for learning at equilibrium
Equilibrium systems are a powerful way to express neural computations. A...
research
∙
05/06/2022
Beyond backpropagation: implicit gradients for bilevel optimization
This paper reviews gradient-based techniques to solve bilevel optimizati...
research
∙
04/14/2022
Minimizing Control for Credit Assignment with Strong Feedback
The success of deep learning attracted interest in whether the brain lea...
research
∙
10/27/2021
Learning where to learn: Gradient sparsity in meta and continual learning
Finding neural network weights that generalize well from small datasets ...
research
∙
06/15/2021
Credit Assignment in Neural Networks through Deep Feedback Control
The success of deep learning sparked interest in whether the brain learn...
research
∙
04/04/2021
A contrastive rule for meta-learning
Meta-learning algorithms leverage regularities that are present on a set...
research
∙
03/01/2021
Posterior Meta-Replay for Continual Learning
Continual Learning (CL) algorithms have recently received a lot of atten...
research
∙
07/25/2020
Economical ensembles with hypernetworks
Averaging the predictions of many independently trained neural networks ...
research
∙
06/25/2020
A Theoretical Framework for Target Propagation
The success of deep learning, a brain-inspired form of AI, has sparked i...
research
∙
11/15/2019
Ghost Units Yield Biologically Plausible Backprop in Deep Neural Networks
In the past few years, deep learning has transformed artificial intellig...
research
∙
06/03/2019
Continual learning with hypernetworks
Artificial neural networks suffer from catastrophic forgetting when they...
research
∙
10/26/2018
Dendritic cortical microcircuits approximate the backpropagation algorithm
Deep learning has seen remarkable developments over the last years, many...
research
∙
12/30/2017
Dendritic error backpropagation in deep cortical microcircuits
Animal behaviour depends on learning to associate sensory stimuli with t...
research
∙
06/06/2016