High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning

by   Antoine Grosnit, et al.

We introduce a method based on deep metric learning to perform Bayesian optimisation over high-dimensional, structured input spaces using variational autoencoders (VAEs). By extending ideas from supervised deep metric learning, we address a longstanding problem in high-dimensional VAE Bayesian optimisation, namely how to enforce a discriminative latent space as an inductive bias. Importantly, we achieve such an inductive bias using just 1 the available labelled data relative to previous work, highlighting the sample efficiency of our approach. As a theoretical contribution, we present a proof of vanishing regret for our method. As an empirical contribution, we present state-of-the-art results on real-world high-dimensional black-box optimisation problems including property-guided molecule generation. It is the hope that the results presented in this paper can act as a guiding principle for realising effective high-dimensional Bayesian optimisation.


page 21

page 24


Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces

High-dimensional black-box optimisation remains an important yet notorio...

The Effect of Intrinsic Dimension on Metric Learning under Compression

Metric learning aims at finding a suitable distance metric over the inpu...

On random embeddings and their application to optimisation

Random embeddings project high-dimensional spaces to low-dimensional one...

Ordinal Bayesian Optimisation

Bayesian optimisation is a powerful tool to solve expensive black-box pr...

Improving black-box optimization in VAE latent space using decoder uncertainty

Optimization in the latent space of variational autoencoders is a promis...

Hierarchical Annotation of Images with Two-Alternative-Forced-Choice Metric Learning

Many tasks such as retrieval and recommendations can significantly benef...

Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?

Learning decompositions of expensive-to-evaluate black-box functions pro...

Please sign up or login with your details

Forgot password? Click here to reset