DeepAI AI Chat
Log In Sign Up

Switch Spaces: Learning Product Spaces with Sparse Gating

by   Shuai Zhang, et al.

Learning embedding spaces of suitable geometry is critical for representation learning. In order for learned representations to be effective and efficient, it is ideal that the geometric inductive bias aligns well with the underlying structure of the data. In this paper, we propose Switch Spaces, a data-driven approach for learning representations in product space. Specifically, product spaces (or manifolds) are spaces of mixed curvature, i.e., a combination of multiple euclidean and non-euclidean (hyperbolic, spherical) manifolds. To this end, we introduce sparse gating mechanisms that learn to choose, combine and switch spaces, allowing them to be switchable depending on the input data with specialization. Additionally, the proposed method is also efficient and has a constant computational complexity regardless of the model size. Experiments on knowledge graph completion and item recommendations show that the proposed switch space achieves new state-of-the-art performances, outperforming pure product spaces and recently proposed task-specific models.


Improving Heterogeneous Graph Learning with Weighted Mixed-Curvature Product Manifold

In graph representation learning, it is important that the complex geome...

Mixed-curvature Variational Autoencoders

It has been shown that using geometric spaces with non-zero curvature in...

Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach

Matrix manifolds, such as manifolds of Symmetric Positive Definite (SPD)...

Overlaying Spaces and Practical Applicability of Complex Geometries

Recently, non-Euclidean spaces became popular for embedding structured d...

Neural Embedding: Learning the Embedding of the Manifold of Physics Data

In this paper, we present a method of embedding physics data manifolds w...

Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning

Real-world graphs naturally exhibit hierarchical or cyclical structures ...

SINCERE: Sequential Interaction Networks representation learning on Co-Evolving RiEmannian manifolds

Sequential interaction networks (SIN) have been commonly adopted in many...