Robust angle-based transfer learning in high dimensions

10/23/2022
by   Tian Gu, et al.
0

Transfer learning aims to improve the performance of a target model by leveraging data from related source populations. It is known to be especially helpful in cases with insufficient target data. In this paper, we study the problem of how to train a high-dimensional ridge regression model with limited target data and existing models trained in heterogeneous source populations. We consider a practical setting where only the source model parameters are accessible, instead of the individual-level source data. Under the setting with only one source model, we propose a novel flexible angle-based transfer learning (angleTL) method, which leverages the concordance between the source and the target model parameters. We show that angleTL unifies several benchmark methods by construction, including the target-only model trained using target data alone, the source model trained using the source data, and the distance-based transfer learning method that incorporates the source model to the target training by penalizing the difference between the target and source model parameters measured by the L_2 norm. We also provide algorithms to effectively incorporate multiple source models accounting for the fact that some source models may be more helpful than others. Our high-dimensional asymptotic analysis provides interpretations and insights regarding when a source model can be helpful to the target model, and demonstrates the superiority of angleTL over other benchmark methods. We perform extensive simulation studies to validate our theoretical conclusions and show the feasibility of applying angleTL to transfer existing genetic risk prediction models across multiple biobanks.

READ FULL TEXT
research
09/12/2023

Distributionally Robust Transfer Learning

Many existing transfer learning methods rely on leveraging information f...
research
08/10/2022

Doubly Robust Augmented Model Accuracy Transfer Inference with High Dimensional Features

Due to label scarcity and covariate shift happening frequently in real-w...
research
07/01/2023

Unified Transfer Learning Models for High-Dimensional Linear Regression

Transfer learning plays a key role in modern data analysis when: (1) the...
research
06/28/2023

Transfer Learning with Random Coefficient Ridge Regression

Ridge regression with random coefficients provides an important alternat...
research
11/27/2020

Randomized Transferable Machine

Feature-based transfer is one of the most effective methodologies for tr...
research
11/29/2022

Transfer Learning with Uncertainty Quantification: Random Effect Calibration of Source to Target (RECaST)

Transfer learning uses a data model, trained to make predictions or infe...
research
02/23/2022

A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality

We study the problem of transfer learning, observing that previous effor...

Please sign up or login with your details

Forgot password? Click here to reset