Model-Protected Multi-Task Learning

by   Jian Liang, et al.

Multi-task learning (MTL) refers to the paradigm of learning multiple related tasks together. By contrast, single-task learning (STL) learns each individual task independently. MTL often leads to better trained models because they can leverage the commonalities among related tasks. However, because MTL algorithms will "transmit" information on different models across different tasks, MTL poses a potential security risk. Specifically, an adversary may participate in the MTL process through a participating task, thereby acquiring the model information for another task. Previously proposed privacy-preserving MTL methods protect data instances rather than models, and some of them may underperform in comparison with STL methods. In this paper, we propose a privacy-preserving MTL framework to prevent the information on each model from leaking to other models based on a perturbation of the covariance matrix of the model matrix, and we study two popular MTL approaches for instantiation, namely, MTL approaches for learning the low-rank and group-sparse patterns of the model matrix. Our methods are built upon tools for differential privacy. Privacy guarantees and utility bounds are provided. Heterogeneous privacy budgets are considered. Our algorithms can be guaranteed not to underperform comparing with STL methods. Experiments demonstrate that our algorithms outperform existing privacy-preserving MTL methods on the proposed model-protection problem.


page 11

page 15


Multi-Trigger-Key: Towards Multi-Task Privacy Preserving In Deep Learning

Deep learning-based Multi-Task Classification (MTC) is widely used in ap...

Privacy-Preserving Matrix Factorization for Recommendation Systems using Gaussian Mechanism

Building a recommendation system involves analyzing user data, which can...

A Privacy Preserving Method with a Random Orthogonal Matrix for ConvMixer Models

In this paper, a privacy preserving image classification method is propo...

A general cipher for individual data anonymization

Over the years, the literature on individual data anonymization has burg...

General Confidentiality and Utility Metrics for Privacy-Preserving Data Publishing Based on the Permutation Model

Anonymization for privacy-preserving data publishing, also known as stat...

De-identification of Privacy-related Entities in Job Postings

De-identification is the task of detecting privacy-related entities in t...

Privacy Preserving Identification Using Sparse Approximation with Ambiguization

In this paper, we consider a privacy preserving encoding framework for i...

Please sign up or login with your details

Forgot password? Click here to reset