research
∙
07/06/2023
Focused Transformer: Contrastive Training for Context Scaling
Large language models have an exceptional capability to incorporate new ...
share
research
∙
06/27/2023
Length Generalization in Arithmetic Transformers
We examine how transformers cope with two challenges: learning basic int...
research
∙
06/02/2023
Evaluating Language Models for Mathematics through Interactions
The standard methodology of evaluating large language models (LLMs) base...
research
∙
05/24/2023
Lexinvariant Language Models
Token embeddings, a mapping from discrete lexical symbols to continuous ...
research
∙
03/08/2023
Magnushammer: A Transformer-based Approach to Premise Selection
Premise selection is a fundamental problem of automated theorem proving....
research
∙
11/18/2022
Path Independent Equilibrium Models Can Better Exploit Test-Time Computation
Designing networks capable of attaining better performance with an incre...
research
∙
11/16/2022
Holistic Evaluation of Language Models
Language models (LMs) are becoming the foundation for almost all major l...
research
∙
10/21/2022
Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs
The formalization of existing mathematical proofs is a notoriously diffi...
research
∙
07/21/2022
Language Model Cascades
Prompted models have demonstrated impressive few-shot learning abilities...
research
∙
07/11/2022
Exploring Length Generalization in Large Language Models
The ability to extrapolate from short problem instances to longer ones i...
research
∙
06/29/2022
Solving Quantitative Reasoning Problems with Language Models
Language models have achieved remarkable performance on a wide range of ...
research
∙
06/21/2022
Insights into Pre-training via Simpler Synthetic Tasks
Pre-training produces representations that are effective for a wide rang...
research
∙
06/01/2022
Fast and Precise: Adjusting Planning Horizon with Adaptive Subgoal Search
Complex reasoning problems contain states that vary in the computational...
research
∙
05/25/2022
Autoformalization with Large Language Models
Autoformalization is the process of automatically translating from natur...
research
∙
05/22/2022
Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers
In theorem proving, the task of selecting useful premises from a large l...
research
∙
03/28/2022
STaR: Bootstrapping Reasoning With Reasoning
Generating step-by-step "chain-of-thought" rationales improves language ...
research
∙
03/16/2022
Memorizing Transformers
Language models typically need to be trained or finetuned in order to ac...
research
∙
03/11/2022
Block-Recurrent Transformers
We introduce the Block-Recurrent Transformer, which applies a transforme...
research
∙
10/26/2021
Hierarchical Transformers Are More Efficient Language Models
Transformer models yield impressive results on many NLP and sequence mod...
research
∙
08/27/2021
Learning to Give Checkable Answers with Prover-Verifier Games
Our ability to know when to trust the decisions made by machine learning...
research
∙
08/25/2021
Subgoal Search For Complex Reasoning Tasks
Humans excel in solving complex reasoning tasks through a mental process...
research
∙
02/24/2021
Nonlinear Invariant Risk Minimization: A Causal Approach
Due to spurious correlations, machine learning systems often fail to gen...
research
∙
02/11/2021
Proof Artifact Co-training for Theorem Proving with Language Models
Labeled data for imitation learning of theorem proving in large librarie...
research
∙
01/15/2021
LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning
While designing inductive bias in neural architectures has been widely s...
research
∙
07/08/2020
The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoning
In this work, we focus on an analogical reasoning task that contains ric...
research
∙
07/07/2020
Learning Branching Heuristics for Propositional Model Counting
Propositional model counting or #SAT is the problem of computing the num...
research
∙
07/06/2020
INT: An Inequality Benchmark for Evaluating Generalization in Theorem Proving
In learning-assisted theorem proving, one of the most critical challenge...
research
∙
06/13/2020
Modelling High-Level Mathematical Reasoning in Mechanised Declarative Proofs
Mathematical proofs can be mechanised using proof assistants to eliminat...
research
∙
06/04/2019
Options as responses: Grounding behavioural hierarchies in multi-agent RL
We propose a novel hierarchical agent architecture for multi-agent reinf...
research
∙
03/07/2019
Concurrent Meta Reinforcement Learning
State-of-the-art meta reinforcement learning algorithms typically assume...
research
∙
02/12/2019
ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning
Sparse reward is one of the most challenging problems in reinforcement l...
research
∙
03/06/2018
Understanding Short-Horizon Bias in Stochastic Meta-Optimization
Careful tuning of the learning rate, or even schedules thereof, can be c...
research
∙
03/03/2018
Some Considerations on Learning to Explore via Meta-Reinforcement Learning
We consider the problem of exploration in meta reinforcement learning. T...
research
∙
01/17/2018
An Empirical Analysis of Proximal Policy Optimization with Kronecker-factored Natural Gradients
In this technical report, we consider an approach that combines the PPO ...
research
∙
08/17/2017
Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
In this work, we propose to apply trust region optimization to deep rein...
research
∙
03/27/2017
Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference
We propose a simple and general variant of the standard reparameterized ...
research
∙
11/14/2016
On the Quantitative Analysis of Decoder-Based Generative Models
The past several years have seen remarkable progress in generative model...
research
∙
05/23/2016
Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations
We investigate the parameter-space geometry of recurrent neural networks...
research
∙
02/26/2016
Architectural Complexity Measures of Recurrent Neural Networks
In this paper, we systematically analyze the connecting architectures of...
research
∙
09/19/2015