Envelopes and principal component regression

by   Xin Zhang, et al.

Envelope methods offer targeted dimension reduction for various models. The overarching goal is to improve efficiency in multivariate parameter estimation by projecting the data onto a lower-dimensional subspace known as the envelope. Envelope approaches have advantages in analyzing data with highly correlated variables, but their iterative Grassmannian optimization algorithms do not scale very well with ultra high-dimensional data. While the connections between envelopes and partial least squares in multivariate linear regression have promoted recent progress in high-dimensional studies of envelopes, we propose a more straightforward way of envelope modeling from a novel principal components regression perspective. The proposed procedure, Non-Iterative Envelope Component Estimation (NIECE), has excellent computational advantages over the iterative Grassmannian optimization alternatives in high dimensions. We develop a unified NIECE theory that bridges the gap between envelope methods and principal components in regression. The new theoretical insights also shed light on the envelope subspace estimation error as a function of eigenvalue gaps of two symmetric positive definite matrices used in envelope modeling. We apply the new theory and algorithm to several envelope models, including response and predictor reduction in multivariate linear models, logistic regression, and Cox proportional hazard model. Simulations and illustrative data analysis show the potential for NIECE to improve standard methods in linear and generalized linear models significantly.


page 1

page 2

page 3

page 4


Sparse principal component regression for generalized linear models

Principal component regression (PCR) is a widely used two-stage procedur...

Envelopes for multivariate linear regression with linearly constrained coefficients

A constrained multivariate linear model is a multivariate linear model w...

Sufficient principal component regression for pattern discovery in transcriptomic data

Methods for global measurement of transcript abundance such as microarra...

Functional Principal Subspace Sampling for Large Scale Functional Data Analysis

Functional data analysis (FDA) methods have computational and theoretica...

Nonparametric principal subspace regression

In scientific applications, multivariate observations often come in tand...

Unsupervised Bump Hunting Using Principal Components

Principal Components Analysis is a widely used technique for dimension r...

Kernel Additive Principal Components

Additive principal components (APCs for short) are a nonlinear generaliz...

Please sign up or login with your details

Forgot password? Click here to reset