Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers

by   Albert Q. Jiang, et al.
University of Cambridge
University of Warsaw
Zimbra, Inc.

In theorem proving, the task of selecting useful premises from a large library to unlock the proof of a given conjecture is crucially important. This presents a challenge for all theorem provers, especially the ones based on language models, due to their relative inability to reason over huge volumes of premises in text form. This paper introduces Thor, a framework integrating language models and automated theorem provers to overcome this difficulty. In Thor, a class of methods called hammers that leverage the power of automated theorem provers are used for premise selection, while all other tasks are designated to language models. Thor increases a language model's success rate on the PISA dataset from 39% to 57%, while solving 8.2% of problems neither language models nor automated theorem provers are able to solve on their own. Furthermore, with a significantly smaller computational budget, Thor can achieve a success rate on the MiniF2F dataset that is on par with the best existing methods. Thor can be instantiated for the majority of popular interactive theorem provers via a straightforward protocol we provide.


page 1

page 2

page 3

page 4


Generative Language Modeling for Automated Theorem Proving

We explore the application of transformer-based language models to autom...

Proof Artifact Co-training for Theorem Proving with Language Models

Labeled data for imitation learning of theorem proving in large librarie...

LeanDojo: Theorem Proving with Retrieval-Augmented Language Models

Large language models (LLMs) have shown promise in proving formal theore...

OCR Language Models with Custom Vocabularies

Language models are useful adjuncts to optical models for producing accu...

Magnushammer: A Transformer-based Approach to Premise Selection

Premise selection is a fundamental problem of automated theorem proving....

Learning distributed representations of graphs with Geo2DR

We present Geo2DR, a Python library for unsupervised learning on graph-s...

MaLeS: A Framework for Automatic Tuning of Automated Theorem Provers

MaLeS is an automatic tuning framework for automated theorem provers. It...

Please sign up or login with your details

Forgot password? Click here to reset