Measuring the Mixing of Contextual Information in the Transformer

by   Javier Ferrando, et al.
Universitat Politècnica de Catalunya

The Transformer architecture aggregates input information through the self-attention mechanism, but there is no clear understanding of how this information is mixed across the entire model. Additionally, recent works have demonstrated that attention weights alone are not enough to describe the flow of information. In this paper, we consider the whole attention block – multi-head attention, residual connection, and layer normalization – and define a metric to measure token-to-token interactions within each layer, considering the characteristics of the representation space. Then, we aggregate layer-wise interpretations to provide input attribution scores for model predictions. Experimentally, we show that our method, ALTI (Aggregation of Layer-wise Token-to-token Interactions), provides faithful explanations and outperforms similar aggregation methods.


page 9

page 10

page 11

page 13


Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

The great success of Transformer-based models benefits from the powerful...

Quantifying Context Mixing in Transformers

Self-attention weights and their transformed variants have been the main...

Incorporating Residual and Normalization Layers into Analysis of Masked Language Models

Transformer architecture has become ubiquitous in the natural language p...

Interpretable Rumor Detection in Microblogs by Attending to User Interactions

We address rumor detection by learning to differentiate between the comm...

DecompX: Explaining Transformers Decisions by Propagating Token Decomposition

An emerging solution for explaining Transformer-based models is to use v...

GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers

There has been a growing interest in interpreting the underlying dynamic...

Convolution-enhanced Evolving Attention Networks

Attention-based neural networks, such as Transformers, have become ubiqu...

Code Repositories


Measuring the Mixing of Contextual Information in the Transformer

view repo

Please sign up or login with your details

Forgot password? Click here to reset