Prayatul Matrix: A Direct Comparison Approach to Evaluate Performance of Supervised Machine Learning Models

by   Anupam Biswas, et al.

Performance comparison of supervised machine learning (ML) models are widely done in terms of different confusion matrix based scores obtained on test datasets. However, a dataset comprises several instances having different difficulty levels. Therefore, it is more logical to compare effectiveness of ML models on individual instances instead of comparing scores obtained for the entire dataset. In this paper, an alternative approach is proposed for direct comparison of supervised ML models in terms of individual instances within the dataset. A direct comparison matrix called Prayatul Matrix is introduced, which accounts for comparative outcome of two ML algorithms on different instances of a dataset. Five different performance measures are designed based on prayatul matrix. Efficacy of the proposed approach as well as designed measures is analyzed with four classification techniques on three datasets. Also analyzed on four large-scale complex image datasets with four deep learning models namely ResNet50V2, MobileNetV2, EfficientNet, and XceptionNet. Results are evident that the newly designed measure are capable of giving more insight about the comparing ML algorithms, which were impossible with existing confusion matrix based scores like accuracy, precision and recall.


page 6

page 7


Prasatul Matrix: A Direct Comparison Approach for Analyzing Evolutionary Optimization Algorithms

The performance of individual evolutionary optimization algorithms is mo...

Machine Learning Pipeline for Pulsar Star Dataset

This work brings together some of the most common machine learning (ML) ...

Towards Demystifying Serverless Machine Learning Training

The appeal of serverless (FaaS) has triggered a growing interest on how ...

Decoding machine learning benchmarks

Despite the availability of benchmark machine learning (ML) repositories...

Characterizing instance hardness in classification and regression problems

Some recent pieces of work in the Machine Learning (ML) literature have ...

LS-CAT: A Large-Scale CUDA AutoTuning Dataset

The effectiveness of Machine Learning (ML) methods depend on access to l...

Characterizing multiple instance datasets

In many pattern recognition problems, a single feature vector is not suf...

Please sign up or login with your details

Forgot password? Click here to reset