Exploring Resiliency to Natural Image Corruptions in Deep Learning using Design Diversity

by   Rafael Rosales, et al.

In this paper, we investigate the relationship between diversity metrics, accuracy, and resiliency to natural image corruptions of Deep Learning (DL) image classifier ensembles. We investigate the potential of an attribution-based diversity metric to improve the known accuracy-diversity trade-off of the typical prediction-based diversity. Our motivation is based on analytical studies of design diversity that have shown that a reduction of common failure modes is possible if diversity of design choices is achieved. Using ResNet50 as a comparison baseline, we evaluate the resiliency of multiple individual DL model architectures against dataset distribution shifts corresponding to natural image corruptions. We compare ensembles created with diverse model architectures trained either independently or through a Neural Architecture Search technique and evaluate the correlation of prediction-based and attribution-based diversity to the final ensemble accuracy. We evaluate a set of diversity enforcement heuristics based on negative correlation learning to assess the final ensemble resilience to natural image corruptions and inspect the resulting prediction, activation, and attribution diversity. Our key observations are: 1) model architecture is more important for resiliency than model size or model accuracy, 2) attribution-based diversity is less negatively correlated to the ensemble accuracy than prediction-based diversity, 3) a balanced loss function of individual and ensemble accuracy creates more resilient ensembles for image natural corruptions, 4) architecture diversity produces more diversity in all explored diversity metrics: predictions, attributions, and activations.


page 3

page 7


Neural Ensemble Search for Performant and Calibrated Predictions

Ensembles of neural networks achieve superior performance compared to st...

The Shooting Regressor; Randomized Gradient-Based Ensembles

An ensemble method is introduced that utilizes randomization and loss fu...

Promoting High Diversity Ensemble Learning with EnsembleBench

Ensemble learning is gaining renewed interests in recent years. This pap...

Deep Negative Correlation Classification

Ensemble learning serves as a straightforward way to improve the perform...

Exploring new ways: Enforcing representational dissimilarity to learn new features and reduce error consistency

Independently trained machine learning models tend to learn similar feat...

Diversity and degrees of freedom in regression ensembles

Ensemble methods are a cornerstone of modern machine learning. The perfo...

Ensembles of Locally Independent Prediction Models

Many ensemble methods encourage their constituent models to be diverse, ...

Please sign up or login with your details

Forgot password? Click here to reset