Disentangling Embedding Spaces with Minimal Distributional Assumptions

by   Tobias Leemann, et al.

Interest in understanding and factorizing learned embedding spaces is growing. For instance, recent concept-based explanation techniques analyze a machine learning model in terms of interpretable latent components. Such components have to be discovered in the model's embedding space, e.g., through independent component analysis (ICA) or modern disentanglement learning techniques. While these unsupervised approaches offer a sound formal framework, they either require access to a data generating function or impose rigid assumptions on the data distribution, such as independence of components, that are often violated in practice. In this work, we link conceptual explainability for vision models with disentanglement learning and ICA. This enables us to provide first theoretical results on how components can be identified without requiring any distributional assumptions. From these insights, we derive the disjoint attributions (DA) concept discovery method that is applicable to a broader class of problems than current approaches but yet possesses a formal identifiability guarantee. In an extensive comparison against component analysis and over 300 state-of-the-art disentanglement models, DA stably maintains superior performance, even under varying distributions and correlation strengths.


page 2

page 9

page 14

page 16

page 17

page 18

page 21


Independent Component Analysis based on multiple data-weighting

Independent Component Analysis (ICA) - one of the basic tools in data an...

Domain adaptation under structural causal models

Domain adaptation (DA) arises as an important problem in statistical mac...

Emblaze: Illuminating Machine Learning Representations through Interactive Comparison of Embedding Spaces

Modern machine learning techniques commonly rely on complex, high-dimens...

Few-shot Domain Adaptation by Causal Mechanism Transfer

We study few-shot supervised domain adaptation (DA) for regression probl...

An analysis on the use of autoencoders for representation learning: fundamentals, learning task case studies, explainability and challenges

In many machine learning tasks, learning a good representation of the da...

Function Classes for Identifiable Nonlinear Independent Component Analysis

Unsupervised learning of latent variable models (LVMs) is widely used to...

Eigen component analysis: A quantum theory incorporated machine learning technique to find linearly maximum separable components

For a linear system, the response to a stimulus is often superposed by i...

Please sign up or login with your details

Forgot password? Click here to reset