Developing Bug-Free Machine Learning Systems With Formal Mathematics

06/26/2017
by   Daniel Selsam, et al.
0

Noisy data, non-convex objectives, model misspecification, and numerical instability can all cause undesired behaviors in machine learning systems. As a result, detecting actual implementation errors can be extremely difficult. We demonstrate a methodology in which developers use an interactive proof assistant to both implement their system and to state a formal theorem defining what it means for their system to be correct. The process of proving this theorem interactively in the proof assistant exposes all implementation errors since any error in the program would cause the proof to fail. As a case study, we implement a new system, Certigrad, for optimizing over stochastic computation graphs, and we generate a formal (i.e. machine-checkable) proof that the gradients sampled by the system are unbiased estimates of the true mathematical gradients. We train a variational autoencoder using Certigrad and find the performance comparable to training the same model in TensorFlow.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/23/2023

A Formal Proof of the Strong Normalization Theorem for System T in Agda

We present a framework for the formal meta-theory of lambda calculi in f...
research
11/01/2019

A Formal Proof of PAC Learnability for Decision Stumps

We present a machine-checked, formal proof of PAC learnability of the co...
research
07/14/2020

Verification of ML Systems via Reparameterization

As machine learning is increasingly used in essential systems, it is imp...
research
04/05/2023

Verifying Quantum Phase Estimation (QPE) using Prove-It

The general-purpose interactive theorem-proving assistant called Prove-I...
research
02/11/2021

Proof Artifact Co-training for Theorem Proving with Language Models

Labeled data for imitation learning of theorem proving in large librarie...
research
01/31/2016

Reliability of Checking an Answer Given by a Mathematical Expression in Interactive Learning Systems

In this article we address the problem of automatic answer checking in i...
research
04/12/2021

Online Machine Learning Techniques for Coq: A Comparison

We present a comparison of several online machine learning techniques fo...

Please sign up or login with your details

Forgot password? Click here to reset