Efficiently Learning the Graph for Semi-supervised Learning

by   Dravyansh Sharma, et al.

Computational efficiency is a major bottleneck in using classic graph-based approaches for semi-supervised learning on datasets with a large number of unlabeled examples. Known techniques to improve efficiency typically involve an approximation of the graph regularization objective, but suffer two major drawbacks - first the graph is assumed to be known or constructed with heuristic hyperparameter values, second they do not provide a principled approximation guarantee for learning over the full unlabeled dataset. Building on recent work on learning graphs for semi-supervised learning from multiple datasets for problems from the same domain, and leveraging techniques for fast approximations for solving linear systems in the graph Laplacian matrix, we propose algorithms that overcome both the above limitations. We show a formal separation in the learning-theoretic complexity of sparse and dense graph families. We further show how to approximately learn the best graphs from the sparse families efficiently using the conjugate gradient method. Our approach can also be used to learn the graph efficiently online with sub-linear regret, under mild smoothness assumptions. Our online learning results are stated generally, and may be useful for approximate and efficient parameter tuning in other problems. We implement our approach and demonstrate significant (∼10-100x) speedups over prior work on semi-supervised learning with learned graphs on benchmark datasets.


page 1

page 2

page 3

page 4


Algorithms for ℓ_p-based semi-supervised learning on graphs

We develop fast algorithms for solving the variational and game-theoreti...

Incremental Spectral Sparsification for Large-Scale Graph-Based Semi-Supervised Learning

While the harmonic function solution performs well in many semi-supervis...

Data driven algorithms for limited labeled data learning

We consider a novel data driven approach for designing learning algorith...

Online Semi-Supervised Learning on Quantized Graphs

In this paper, we tackle the problem of online semi-supervised learning ...

Improving Robustness of Attention Models on Graphs

Machine learning models that can exploit the inherent structure in data ...

Graph-based Semi-supervised Learning: A Comprehensive Review

Semi-supervised learning (SSL) has tremendous value in practice due to i...

Semi-Supervised Learning on Graphs with Feature-Augmented Graph Basis Functions

For semi-supervised learning on graphs, we study how initial kernels in ...

Please sign up or login with your details

Forgot password? Click here to reset