Prasatul Matrix: A Direct Comparison Approach for Analyzing Evolutionary Optimization Algorithms

by   Anupam Biswas, et al.

The performance of individual evolutionary optimization algorithms is mostly measured in terms of statistics such as mean, median and standard deviation etc., computed over the best solutions obtained with few trails of the algorithm. To compare the performance of two algorithms, the values of these statistics are compared instead of comparing the solutions directly. This kind of comparison lacks direct comparison of solutions obtained with different algorithms. For instance, the comparison of best solutions (or worst solution) of two algorithms simply not possible. Moreover, ranking of algorithms is mostly done in terms of solution quality only, despite the fact that the convergence of algorithm is also an important factor. In this paper, a direct comparison approach is proposed to analyze the performance of evolutionary optimization algorithms. A direct comparison matrix called Prasatul Matrix is prepared, which accounts direct comparison outcome of best solutions obtained with two algorithms for a specific number of trials. Five different performance measures are designed based on the prasatul matrix to evaluate the performance of algorithms in terms of Optimality and Comparability of solutions. These scores are utilized to develop a score-driven approach for comparing performance of multiple algorithms as well as for ranking both in the grounds of solution quality and convergence analysis. Proposed approach is analyzed with six evolutionary optimization algorithms on 25 benchmark functions. A non-parametric statistical analysis, namely Wilcoxon paired sum-rank test is also performed to verify the outcomes of proposed direct comparison approach.


Prayatul Matrix: A Direct Comparison Approach to Evaluate Performance of Supervised Machine Learning Models

Performance comparison of supervised machine learning (ML) models are wi...

On the Fair Comparison of Optimization Algorithms in Different Machines

An experimental comparison of two or more optimization algorithms requir...

SELECTOR: Selecting a Representative Benchmark Suite for Reproducible Statistical Comparison

Fair algorithm evaluation is conditioned on the existence of high-qualit...

Compact Optimization Algorithms with Re-sampled Inheritance

Compact optimization algorithms are a class of Estimation of Distributio...

The Cone epsilon-Dominance: An Approach for Evolutionary Multiobjective Optimization

We propose the cone epsilon-dominance approach to improve convergence an...

Sample size estimation for power and accuracy in the experimental comparison of algorithms

Experimental comparisons of performance represent an important aspect of...

Please sign up or login with your details

Forgot password? Click here to reset