Sequential Model-Based Ensemble Optimization

by   Alexandre Lacoste, et al.

One of the most tedious tasks in the application of machine learning is model selection, i.e. hyperparameter selection. Fortunately, recent progress has been made in the automation of this process, through the use of sequential model-based optimization (SMBO) methods. This can be used to optimize a cross-validation performance of a learning algorithm over the value of its hyperparameters. However, it is well known that ensembles of learned models almost consistently outperform a single model, even if properly selected. In this paper, we thus propose an extension of SMBO methods that automatically constructs such ensembles. This method builds on a recently proposed ensemble construction paradigm known as agnostic Bayesian learning. In experiments on 22 regression and 39 classification data sets, we confirm the success of this proposed approach, which is able to outperform model selection with SMBO.


page 1

page 2

page 3

page 4


Systematic Ensemble Learning for Regression

The motivation of this work is to improve the performance of standard st...

Visual Integration of Data and Model Space in Ensemble Learning

Ensembles of classifier models typically deliver superior performance an...

Optimizing Unlicensed Coexistence Network Performance Through Data Learning

Unlicensed LTE-WiFi coexistence networks are undergoing consistent densi...

Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach

Tree ensembles, such as random forests and boosted trees, are renowned f...

Agreement-based Learning

Model selection is a problem that has occupied machine learning research...

Optimizing model-agnostic Random Subspace ensembles

This paper presents a model-agnostic ensemble approach for supervised le...

EEE, Remediating the failure of machine learning models via a network-based optimization patch

A network-based optimization approach, EEE, is proposed for the purpose ...

Code Repositories


Spearmint Salad is a combination of Spearmint and Agnostic Bayes to create an ensemble of predictors over the hyperparameter space while using the fast hyperparameter search approach of Spearmint.

view repo

Please sign up or login with your details

Forgot password? Click here to reset