Policy Search with High-Dimensional Context Variables

by   Voot Tangkaratt, et al.

Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application of unsupervised dimensionality reduction methods to the context variables, such as principal component analysis, is insufficient as task-relevant input may be ignored. In this paper, we propose a contextual policy search method in the model-based relative entropy stochastic search framework with integrated dimensionality reduction. We learn a model of the reward that is locally quadratic in both the policy parameters and the context variables. Furthermore, we perform supervised linear dimensionality reduction on the context variables by nuclear norm regularization. The experimental results show that the proposed method outperforms naive dimensionality reduction via principal component analysis and a state-of-the-art contextual policy search method.


Prescriptive PCA: Dimensionality Reduction for Two-stage Stochastic Optimization

In this paper, we consider the alignment between an upstream dimensional...

Visualization of High-dimensional Scalar Functions Using Principal Parameterizations

Insightful visualization of multidimensional scalar fields, in particula...

A Unifying Probabilistic Perspective for Spectral Dimensionality Reduction: Insights and New Models

We introduce a new perspective on spectral dimensionality reduction whic...

MASS-UMAP: Fast and accurate analog ensemble search in weather radar archive

The use of analogs - similar weather patterns - for weather forecasting ...

Lazy stochastic principal component analysis

Stochastic principal component analysis (SPCA) has become a popular dime...

Regression-based music emotion prediction using triplet neural networks

In this paper, we adapt triplet neural networks (TNNs) to a regression t...

Dimensionality Reduction and Wasserstein Stability for Kernel Regression

In a high-dimensional regression framework, we study consequences of the...

Please sign up or login with your details

Forgot password? Click here to reset