Collaborative and Privacy-Preserving Machine Teaching via Consensus Optimization

by   Yufei Han, et al.

In this work, we define a collaborative and privacy-preserving machine teaching paradigm with multiple distributed teachers. We focus on consensus super teaching. It aims at organizing distributed teachers to jointly select a compact while informative training subset from data hosted by the teachers to make a learner learn better. The challenges arise from three perspectives. First, the state-of-the-art pool-based super teaching method applies mixed-integer non-linear programming (MINLP) which does not scale well to very large data sets. Second, it is desirable to restrict data access of the teachers to only their own data during the collaboration stage to mitigate privacy leaks. Finally, the teaching collaboration should be communication-efficient since large communication overheads can cause synchronization delays between teachers. To address these challenges, we formulate collaborative teaching as a consensus and privacy-preserving optimization process to minimize teaching risk. We theoretically demonstrate the necessity of collaboration between teachers for improving the learner's learning. Furthermore, we show that the proposed method enjoys a similar property as the Oracle property of adaptive Lasso. The empirical study illustrates that our teaching method can deliver significantly more accurate teaching results with high speed, while the non-collaborative MINLP-based super teaching becomes prohibitively expensive to compute.


Teacher Improves Learning by Selecting a Training Subset

We call a learner super-teachable if a teacher can trim down an iid trai...

Robust Federated Training via Collaborative Machine Teaching using Trusted Instances

Federated learning performs distributed model training using local data ...

SoK: Privacy-Preserving Collaborative Tree-based Model Learning

Tree-based models are among the most efficient machine learning techniqu...

PP-MARL: Efficient Privacy-Preserving MARL for Cooperative Intelligence in Communication

Artificial intelligence (AI) has been introduced in communication networ...

Unlearn What You Have Learned: Adaptive Crowd Teaching with Exponentially Decayed Memory Learners

With the increasing demand for large amount of labeled data, crowdsourci...

Data collaboration analysis for distributed datasets

In this paper, we propose a data collaboration analysis method for distr...

Challenging but Full of Opportunities: Teachers' Perspectives on Programming in Primary Schools

The widespread establishment of computational thinking in school curricu...

Please sign up or login with your details

Forgot password? Click here to reset