Neural Bug Finding: A Study of Opportunities and Challenges

by   Andrew Habib, et al.

Static analysis is one of the most widely adopted techniques to find software bugs before code is put in production. Designing and implementing effective and efficient static analyses is difficult and requires high expertise, which results in only a few experts able to write such analyses. This paper explores the opportunities and challenges of an alternative way of creating static bug detectors: neural bug finding. The basic idea is to formulate bug detection as a classification problem, and to address this problem with neural networks trained on examples of buggy and non-buggy code. We systematically study the effectiveness of this approach based on code examples labeled by a state-of-the-art, static bug detector. Our results show that neural bug finding is surprisingly effective for some bug patterns, sometimes reaching a precision and recall of over 80 properties obvious to a traditional analysis. A qualitative analysis of the results provides insights into why neural bug finders sometimes work and sometimes do not work. We also identify pitfalls in selecting the code examples used to train and validate neural bug finders, and propose an algorithm for selecting effective training data.


page 1

page 2

page 3

page 4


Find Bugs in Static Bug Finders

Static bug finders have been widely-adopted by developers to find bugs i...

DeepBugs: A Learning Approach to Name-based Bug Detection

Natural language elements in source code, e.g., the names of variables a...

Bug Hunters' Perspectives on the Challenges and Benefits of the Bug Bounty Ecosystem

Although researchers have characterized the bug-bounty ecosystem from th...

A True Positives Theorem for a Static Race Detector - Extended Version

RacerD is a static race detector that has been proven to be effective in...

Learning a Static Bug Finder from Data

Static analysis is an effective technique to catch bugs early when they ...

On Distribution Shift in Learning-based Bug Detectors

Deep learning has recently achieved initial success in program analysis ...

XCheck: a Simple, Effective and Extensible Bug Finder using micro-grammar

We propose a simple and effective bug finder, XCheck, which is a proof o...

Please sign up or login with your details

Forgot password? Click here to reset