Evaluation of Confidence-based Ensembling in Deep Learning Image Classification

by   Rafael Rosales, et al.

Ensembling is a successful technique to improve the performance of machine learning (ML) models. Conf-Ensemble is an adaptation to Boosting to create ensembles based on model confidence instead of model errors to better classify difficult edge-cases. The key idea is to create successive model experts for samples that were difficult (not necessarily incorrectly classified) by the preceding model. This technique has been shown to provide better results than boosting in binary-classification with a small feature space ( 80 features). In this paper, we evaluate the Conf-Ensemble approach in the much more complex task of image classification with the ImageNet dataset (224x224x3 features with 1000 classes). Image classification is an important benchmark for AI-based perception and thus it helps to assess if this method can be used in safety-critical applications using ML ensembles. Our experiments indicate that in a complex multi-label classification task, the expected benefit of specialization on complex input samples cannot be achieved with a small sample set, i.e., a good classifier seems to rely on very complex feature analysis that cannot be well trained on just a limited subset of "difficult samples". We propose an improvement to Conf-Ensemble to increase the number of samples fed to successive ensemble members, and a three-member Conf-Ensemble using this improvement was able to surpass a single model in accuracy, although the amount is not significant. Our findings shed light on the limits of the approach and the non-triviality of harnessing big data.


page 7

page 10


Multiclass Classification with an Ensemble of Binary Classification Deep Networks

Deep neural network classifiers have been used frequently and are effici...

A Novel Online Stacked Ensemble for Multi-Label Stream Classification

As data streams become more prevalent, the necessity for online algorith...

Ensemble learning of diffractive optical networks

A plethora of research advances have emerged in the fields of optics and...

A tutorial on ensembles and deep learning fusion with MNIST as guiding thread: A complex heterogeneous fusion scheme reaching 10 digits error

Ensemble methods have been widely used for improving the results of the ...

A Baseline for Multi-Label Image Classification Using Ensemble Deep CNN

Recent studies on multi-label image classification have been focusing on...

Deep Network Ensemble Learning applied to Image Classification using CNN Trees

Traditional machine learning approaches may fail to perform satisfactori...

Ensemble Models for Detecting Wikidata Vandalism with Stacking - Team Honeyberry Vandalism Detector at WSDM Cup 2017

The WSDM Cup 2017 is a binary classification task for classifying Wikida...

Please sign up or login with your details

Forgot password? Click here to reset