Testing Research Software: A Survey

by   Nasir U. Eisty, et al.

Background: Research software plays an important role in solving real-life problems, empowering scientific innovations, and handling emergency situations. Therefore, the correctness and trustworthiness of research software are of absolute importance. Software testing is an important activity for identifying problematic code and helping to produce high-quality software. However, testing of research software is difficult due to the complexity of the underlying science, relatively unknown results from scientific algorithms, and the culture of the research software community. Aims: The goal of this paper is to better understand current testing practices, identify challenges, and provide recommendations on how to improve the testing process for research software development. Method: We surveyed members of the research software developer community to collect information regarding their knowledge about and use of software testing in their projects. Results: We analysed 120 responses and identified that even though research software developers report they have an average level of knowledge about software testing, they still find it difficult due to the numerous challenges involved. However, there are a number of ways, such as proper training, that can improve the testing process for research software. Conclusions: Testing can be challenging for any type of software. This difficulty is especially present in the development of research software, where software engineering activities are typically given less attention. To produce trustworthy results from research software, there is a need for a culture change so that testing is valued and teams devote appropriate effort to writing and executing tests.


page 8

page 11

page 12

page 13

page 15

page 19


Developers Perception of Peer Code Review in Research Software Development

Background: Research software is software developed by and/or used by re...

Testing Scientific Software: A Systematic Literature Review

Context: Scientific software plays an important role in critical decisio...

Methodology for development of scientific software and test frameworks in function of precision of the expected results

This dissertation focuses on the development process of scientific softw...

RAMPVIS: Towards a New Methodology for Developing Visualisation Capabilities for Large-scale Emergency Responses

The effort for combating the COVID-19 pandemic around the world has resu...

Accessibility Evaluation of Computer Based Tests

Computer-based tests (CBTs) play an important role in the professional c...

Automated and manual testing as part of the research software development process of RCE

Research software is often developed by individual researchers or small ...

Testability First!

The pivotal role of testing in high-quality software production has driv...

Please sign up or login with your details

Forgot password? Click here to reset