Building benchmarking frameworks for supporting replicability and reproducibility: spatial and textual analysis as an example

by   Yingjie Hu, et al.

Replicability and reproducibility (R R) are critical for the long-term prosperity of a scientific discipline. In GIScience, researchers have discussed R R related to different research topics and problems, such as local spatial statistics, digital earth, and metadata (Fotheringham, 2009; Goodchild, 2012; Anselin et al., 2014). This position paper proposes to further support R R by building benchmarking frameworks in order to facilitate the replication of previous research for effective and effcient comparisons of methods and software tools developed for addressing the same or similar problems. Particularly, this paper will use geoparsing, an important research problem in spatial and textual analysis, as an example to explain the values of such benchmarking frameworks.


page 1

page 2


Benchmarking of Java Verification Tools at the Software Verification Competition (SV-COMP)

Empirical evaluation of verification tools by benchmarking is a common m...

Benchmarking in Optimization: Best Practice and Open Issues

This survey compiles ideas and recommendations from more than a dozen re...

Benchmarking Deep Learning Hardware and Frameworks: Qualitative Metrics

Previous survey papers offer knowledge of deep learning hardware devices...

Qualitative Benchmarking of Deep Learning Hardware and Frameworks: Review and Tutorial

Previous survey papers offer knowledge of deep learning hardware devices...

Benchmarking Crimes: An Emerging Threat in Systems Security

Properly benchmarking a system is a difficult and intricate task. Unfort...

LITMUS: An Open Extensible Framework for Benchmarking RDF Data Management Solutions

Developments in the context of Open, Big, and Linked Data have led to an...

Towards Long-term and Archivable Reproducibility

Analysis pipelines commonly use high-level technologies that are popular...

Please sign up or login with your details

Forgot password? Click here to reset