Covariance-free Partial Least Squares: An Incremental Dimensionality Reduction Method

by   Artur Jordao, et al.
Universidade Federal de Minas Gerais

Dimensionality reduction plays an important role in computer vision problems since it reduces computational cost and is often capable of yielding more discriminative data representation. In this context, Partial Least Squares (PLS) has presented notable results in tasks such as image classification and neural network optimization. However, PLS is infeasible on large datasets (e.g., ImageNet) because it requires all the data to be in memory in advance, which is often impractical due to hardware limitations. Additionally, this requirement prevents us from employing PLS on streaming applications where the data are being continuously generated. Motivated by this, we propose a novel incremental PLS, named Covariance-free Incremental Partial Least Squares (CIPLS), which learns a low-dimensional representation of the data using a single sample at a time. In contrast to other state-of-the-art approaches, instead of adopting a partially-discriminative or SGD-based model, we extend Nonlinear Iterative Partial Least Squares (NIPALS) - the standard algorithm used to compute PLS - for incremental processing. Among the advantages of this approach are the preservation of discriminative information across all components, the possibility of employing its score matrices for feature selection, and its computational efficiency. We validate CIPLS on face verification and image classification tasks, where it outperforms several other incremental dimensionality reduction methods. In the context of feature selection, CIPLS achieves comparable results when compared to state-of-the-art techniques.


Partial least squares discriminant analysis: A dimensionality reduction method to classify hyperspectral data

The recent development of more sophisticated spectroscopic methods allow...

High-Dimensional Feature Selection for Genomic Datasets

In the presence of large dimensional datasets that contain many irreleva...

Scalable Derivative-Free Optimization for Nonlinear Least-Squares Problems

Derivative-free - or zeroth-order - optimization (DFO) has gained recent...

Image Classification by Feature Dimension Reduction and Graph based Ranking

Dimensionality reduction (DR) of image features plays an important role ...

Nonlinear Feature Aggregation: Two Algorithms driven by Theory

Many real-world machine learning applications are characterized by a hug...

On the Computation and Applications of Large Dense Partial Correlation Networks

While sparse inverse covariance matrices are very popular for modeling n...

Sentence-Incremental Neural Coreference Resolution

We propose a sentence-incremental neural coreference resolution system w...

Please sign up or login with your details

Forgot password? Click here to reset