Essential guidelines for computational method benchmarking

by   Lukas M. Weber, et al.

In computational biology and other sciences, researchers are frequently faced with a choice between several computational methods for performing data analyses. Benchmarking studies aim to rigorously compare the performance of different methods using well-characterized benchmark datasets, to determine the strengths of each method or to provide recommendations regarding the best choice of method for an analysis. However, benchmarking studies must be carefully designed and implemented to provide accurate and unbiased results. Here, we summarize key practical guidelines and recommendations for performing high-quality benchmarking analyses, based on our own experiences in computational biology.


Guidelines for benchmarking of optimization approaches for fitting mathematical models

Insufficient performance of optimization approaches for fitting of mathe...

Benchmarking in Optimization: Best Practice and Open Issues

This survey compiles ideas and recommendations from more than a dozen re...

On the role of benchmarking data sets and simulations in method comparison studies

Method comparisons are essential to provide recommendations and guidance...

A Benchmarking Study on Vision-Based Grasp Synthesis Algorithms

In this paper, we present a benchmarking study of vision-based grasp syn...

Recommendations for machine learning validation in biology

Modern biology frequently relies on machine learning to provide predicti...

Password Guessers Under a Microscope: An In-Depth Analysis to Inform Deployments

Password guessers are instrumental for assessing the strength of passwor...

TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks

While LLMs have shown great success in understanding and generating text...

Please sign up or login with your details

Forgot password? Click here to reset