Identifying Properties of Real-World Optimisation Problems through a Questionnaire

by   Koen van der Blom, et al.

Optimisation algorithms are commonly compared on benchmarks to get insight into performance differences. However, it is not clear how closely benchmarks match the properties of real-world problems because these properties are largely unknown. This work investigates the properties of real-world problems through a questionnaire to enable the design of future benchmark problems that more closely resemble those found in the real world. The results, while not representative, show that many problems possess at least one of the following properties: they are constrained, deterministic, have only continuous variables, require substantial computation times for both the objectives and the constraints, or allow a limited number of evaluations. Properties like known optimal solutions and analytical gradients are rarely available, limiting the options in guiding the optimisation process. These are all important aspects to consider when designing realistic benchmark problems. At the same time, objective functions are often reported to be black-box and since many problem properties are unknown the design of realistic benchmarks is difficult. To further improve the understanding of real-world problems, readers working on a real-world optimisation problem are encouraged to fill out the questionnaire:


page 1

page 2

page 3

page 4


Towards Realistic Optimization Benchmarks: A Questionnaire on the Properties of Real-World Problems

Benchmarks are a useful tool for empirical performance comparisons. Howe...

EXPObench: Benchmarking Surrogate-based Optimisation Algorithms on Expensive Black-box Functions

Surrogate algorithms such as Bayesian optimisation are especially design...

A Principled Method for the Creation of Synthetic Multi-fidelity Data Sets

Multifidelity and multioutput optimisation algorithms are of active inte...

Bayesian Optimisation Against Climate Change: Applications and Benchmarks

Bayesian optimisation is a powerful method for optimising black-box func...

The Dynamic Travelling Thief Problem: Benchmarks and Performance of Evolutionary Algorithms

Many real-world optimisation problems involve dynamic and stochastic com...

Domain-Agnostic Batch Bayesian Optimization with Diverse Constraints via Bayesian Quadrature

Real-world optimisation problems often feature complex combinations of (...

Please sign up or login with your details

Forgot password? Click here to reset