Wanted: standards for automatic reproducibility of computational experiments

by   Samuel Grayson, et al.
University of Illinois at Urbana-Champaign
Sandia National Laboratories

Those seeking to reproduce a computational experiment often need to manually look at the code to see how to build necessary libraries, configure parameters, find data, and invoke the experiment; it is not automatic. Automatic reproducibility is a more stringent goal, but working towards it would benefit the community. This work discusses a machine-readable language for specifying how to execute a computational experiment. We invite interested stakeholders to discuss this language at https://github.com/charmoniumQ/execution-description .


page 1

page 2

page 3

page 4


Reproducibility Report: Contrastive Learning of Socially-aware Motion Representations

The following paper is a reproducibility report for "Social NCE: Contras...

A Backend Platform for Supporting the Reproducibility of Computational Experiments

In recent years, the research community has raised serious questions abo...

Research Reproducibility as a Survival Analysis

There has been increasing concern within the machine learning community ...

Detecting Inactive Cyberwarriors from Online Forums

The proliferation of misinformation has emerged as a new form of warfare...

Experiments as Code: A Concept for Reproducible, Auditable, Debuggable, Reusable, Scalable Experiments

A common concern in experimental research is the auditability and reprod...

RANG: Reconstructing reproducible R computational environments

A complete declarative description of the computational environment is o...

REPETITA: Repeatable Experiments for Performance Evaluation of Traffic-Engineering Algorithms

In this paper, we propose a pragmatic approach to improve reproducibilit...

Please sign up or login with your details

Forgot password? Click here to reset