SIERRA: A Modular Framework for Research Automation

by   John Harwell, et al.
University of Minnesota

Modern intelligent systems researchers employ the scientific method: they form hypotheses about system behavior, and then run experiments using one or more independent variables to test their hypotheses. We present SIERRA, a novel framework structured around that idea for accelerating research developments and improving reproducibility of results. SIERRA makes it easy to quickly specify the independent variable(s) for an experiment, generate experimental inputs, automatically run the experiment, and process the results to generate deliverables such as graphs and videos. SIERRA provides reproducible automation independent of the execution environment (HPC hardware, real robots, etc.) and targeted platform (arbitrary simulator or real robots), enabling exact experiment replication (up to the limit of the execution environment and platform). It employs a deeply modular approach that allows easy customization and extension of automation for the needs of individual researchers, thereby eliminating manual experiment configuration and result processing via throw-away scripts.


SIERRA: A Modular Framework for Research Automation and Reproducibility

Modern intelligent systems researchers form hypotheses about system beha...

Accelerating Test Automation through a Domain Specific Language

Test automation involves the automatic execution of test scripts instead...

Toward Enabling Reproducibility for Data-Intensive Research using the Whole Tale Platform

Whole Tale is a web-based, open-source platform for...

Globus Automation Services: Research process automation across the space-time continuum

Research process automation–the reliable, efficient, and reproducible ex...

AllenNLP: A Deep Semantic Natural Language Processing Platform

This paper describes AllenNLP, a platform for research on deep learning ...

Improving Reproducible Deep Learning Workflows with DeepDIVA

The field of deep learning is experiencing a trend towards producing rep...

Experiments as Code: A Concept for Reproducible, Auditable, Debuggable, Reusable, Scalable Experiments

A common concern in experimental research is the auditability and reprod...

Please sign up or login with your details

Forgot password? Click here to reset