Monotone Improvement of Information-Geometric Optimization Algorithms with a Surrogate Function

by   Youhei Akimoto, et al.
University of Tsukuba

A surrogate function is often employed to reduce the number of objective function evaluations for optimization. However, the effect of using a surrogate model in evolutionary approaches has not been theoretically investigated. This paper theoretically analyzes the information-geometric optimization framework using a surrogate function. The value of the expected objective function under the candidate sampling distribution is used as the measure of progress of the algorithm. We assume that the surrogate function is maintained so that the population version of the Kendall's rank correlation coefficient between the surrogate function and the objective function under the candidate sampling distribution is greater than or equal to a predefined threshold. We prove that information-geometric optimization using such a surrogate function leads to a monotonic decrease in the expected objective function value if the threshold is sufficiently close to one. The acceptable threshold value is analyzed for the case of the information-geometric optimization instantiated with Gaussian distributions, i.e., the rank-μ update CMA-ES, on a convex quadratic objective function. As an alternative to the Kendall's rank correlation coefficient, we investigate the use of the Pearson correlation coefficient between the weights assigned to candidate solutions based on the objective function and the surrogate function.


page 1

page 2

page 3

page 4


Meta-Model Framework for Surrogate-Based Parameter Estimation in Dynamical Systems

The central task in modeling complex dynamical systems is parameter esti...

Surrogate-based cross-correlation for particle image velocimetry

This paper presents a novel surrogate-based cross-correlation (SBCC) fra...

Error Analysis of Surrogate Models Constructed through Operations on Sub-models

Model-based methods are popular in derivative-free optimization (DFO). I...

Directed particle swarm optimization with Gaussian-process-based function forecasting

Particle swarm optimization (PSO) is an iterative search method that mov...

Information theoretic learning of robust deep representations

We propose a novel objective function for learning robust deep represent...

Objective Improvement in Information-Geometric Optimization

Information-Geometric Optimization (IGO) is a unified framework of stoch...

Active preference learning based on radial basis functions

This paper proposes a method for solving optimization problems in which ...

Please sign up or login with your details

Forgot password? Click here to reset