Neural Manifold Clustering and Embedding

by   Zengyi Li, et al.

Given a union of non-linear manifolds, non-linear subspace clustering or manifold clustering aims to cluster data points based on manifold structures and also learn to parameterize each manifold as a linear subspace in a feature space. Deep neural networks have the potential to achieve this goal under highly non-linear settings given their large capacity and flexibility. We argue that achieving manifold clustering with neural networks requires two essential ingredients: a domain-specific constraint that ensures the identification of the manifolds, and a learning algorithm for embedding each manifold to a linear subspace in the feature space. This work shows that many constraints can be implemented by data augmentation. For subspace feature learning, Maximum Coding Rate Reduction (MCR^2) objective can be used. Putting them together yields Neural Manifold Clustering and Embedding (NMCE), a novel method for general purpose manifold clustering, which significantly outperforms autoencoder-based deep subspace clustering. Further, on more challenging natural image datasets, NMCE can also outperform other algorithms specifically designed for clustering. Qualitatively, we demonstrate that NMCE learns a meaningful and interpretable feature space. As the formulation of NMCE is closely related to several important Self-supervised learning (SSL) methods, we believe this work can help us build a deeper understanding on SSL representation learning.


page 1

page 5

page 6

page 7

page 9

page 10

page 11

page 13


Unsupervised Manifold Linearizing and Clustering

Clustering data lying close to a union of low-dimensional manifolds, wit...

N2D:(Not Too) Deep clustering via clustering the local manifold of an autoencoded embedding

Deep clustering has increasingly been demonstrating superiority over con...

Manifold Learning Benefits GANs

In this paper, we improve Generative Adversarial Networks by incorporati...

Robust Self-Supervised Convolutional Neural Network for Subspace Clustering and Classification

Insufficient capability of existing subspace clustering methods to handl...

Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations

Self-supervised Learning (SSL) provides a strategy for constructing usef...

Multiple View Generation and Classification of Mid-wave Infrared Images using Deep Learning

We propose a novel study of generating unseen arbitrary viewpoints for i...

Manifold Forests: Closing the Gap on Neural Networks

Decision forests (DF), in particular random forests and gradient boostin...

Please sign up or login with your details

Forgot password? Click here to reset