Multi-Scale Prototypical Transformer for Whole Slide Image Classification

by   Saisai Ding, et al.

Whole slide image (WSI) classification is an essential task in computational pathology. Despite the recent advances in multiple instance learning (MIL) for WSI classification, accurate classification of WSIs remains challenging due to the extreme imbalance between the positive and negative instances in bags, and the complicated pre-processing to fuse multi-scale information of WSI. To this end, we propose a novel multi-scale prototypical Transformer (MSPT) for WSI classification, which includes a prototypical Transformer (PT) module and a multi-scale feature fusion module (MFFM). The PT is developed to reduce redundant instances in bags by integrating prototypical learning into the Transformer architecture. It substitutes all instances with cluster prototypes, which are then re-calibrated through the self-attention mechanism of the Trans-former. Thereafter, an MFFM is proposed to fuse the clustered prototypes of different scales, which employs MLP-Mixer to enhance the information communication between prototypes. The experimental results on two public WSI datasets demonstrate that the proposed MSPT outperforms all the compared algorithms, suggesting its potential applications.


page 1

page 2

page 3

page 4


Multi-scale Efficient Graph-Transformer for Whole Slide Image Classification

The multi-scale information among the whole slide images (WSIs) is essen...

Multi-Scale Self-Attention for Text Classification

In this paper, we introduce the prior knowledge, multi-scale structure, ...

Aggregated Text Transformer for Scene Text Detection

This paper explores the multi-scale aggregation strategy for scene text ...

Point Cloud Learning with Transformer

Remarkable performance from Transformer networks in Natural Language Pro...

UHD Image Deblurring via Multi-scale Cubic-Mixer

Currently, transformer-based algorithms are making a splash in the domai...

GridDehazeNet+: An Enhanced Multi-Scale Network with Intra-Task Knowledge Transfer for Single Image Dehazing

We propose an enhanced multi-scale network, dubbed GridDehazeNet+, for s...

Trusted Multi-Scale Classification Framework for Whole Slide Image

Despite remarkable efforts been made, the classification of gigapixels w...

Please sign up or login with your details

Forgot password? Click here to reset