PCA-Boosted Autoencoders for Nonlinear Dimensionality Reduction in Low Data Regimes

by   Muhammad Al-Digeil, et al.

Autoencoders (AE) provide a useful method for nonlinear dimensionality reduction but are ill-suited for low data regimes. Conversely, Principal Component Analysis (PCA) is data-efficient but is limited to linear dimensionality reduction, posing a problem when data exhibits inherent nonlinearity. This presents a challenge in various scientific and engineering domains such as the nanophotonic component design, where data exhibits nonlinear features while being expensive to obtain due to costly real measurements or resource-consuming solutions of partial differential equations. To address this difficulty, we propose a technique that harnesses the best of both worlds: an autoencoder that leverages PCA to perform well on scarce nonlinear data. Specifically, we outline a numerically robust PCA-based initialization of AE, which, together with the parameterized ReLU activation function, allows the training process to start from an exact PCA solution and improve upon it. A synthetic example is presented first to study the effects of data nonlinearity and size on the performance of the proposed method. We then evaluate our method on several nanophotonic component design problems where obtaining useful data is expensive. To demonstrate universality, we also apply it to tasks in other scientific domains: a benchmark breast cancer dataset and a gene expression dataset. We show that our proposed approach is substantially better than both PCA and randomly initialized AE in the majority of low-data regime cases we consider, or at least is comparable to the best of either of the other two methods.


page 9

page 12


Improved Dimensionality Reduction of various Datasets using Novel Multiplicative Factoring Principal Component Analysis (MPCA)

Principal Component Analysis (PCA) is known to be the most widely applie...

Scaled torus principal component analysis

A particularly challenging context for dimensionality reduction is multi...

Low-dimensional Data-based Surrogate Model of a Continuum-mechanical Musculoskeletal System Based on Non-intrusive Model Order Reduction

In recent decades, the main focus of computer modeling has been on suppo...

Dimensionality Reduction via Regression in Hyperspectral Imagery

This paper introduces a new unsupervised method for dimensionality reduc...

Symmetry-Aware Autoencoders: s-PCA and s-nlPCA

Nonlinear principal component analysis (nlPCA) via autoencoders has attr...

AutoQML: Automatic Generation and Training of Robust Quantum-Inspired Classifiers by Using Genetic Algorithms on Grayscale Images

We propose a new hybrid system for automatically generating and training...

Regression-based music emotion prediction using triplet neural networks

In this paper, we adapt triplet neural networks (TNNs) to a regression t...

Please sign up or login with your details

Forgot password? Click here to reset