Adjusting for Chance Clustering Comparison Measures

by   Simone Romano, et al.
The University of Melbourne

Adjusted for chance measures are widely used to compare partitions/clusterings of the same data set. In particular, the Adjusted Rand Index (ARI) based on pair-counting, and the Adjusted Mutual Information (AMI) based on Shannon information theory are very popular in the clustering community. Nonetheless it is an open problem as to what are the best application scenarios for each measure and guidelines in the literature for their usage are sparse, with the result that users often resort to using both. Generalized Information Theoretic (IT) measures based on the Tsallis entropy have been shown to link pair-counting and Shannon IT measures. In this paper, we aim to bridge the gap between adjustment of measures based on pair-counting and measures based on information theory. We solve the key technical challenge of analytically computing the expected value and variance of generalized IT measures. This allows us to propose adjustments of generalized IT measures, which reduce to well known adjusted clustering comparison measures as special cases. Using the theory of generalized IT measures, we are able to propose the following guidelines for using ARI and AMI as external validation indices: ARI should be used when the reference clustering has large equal sized clusters; AMI should be used when the reference clustering is unbalanced and there exist small clusters.


page 1

page 2

page 3

page 4


Understanding partition comparison indices based on counting object pairs

In unsupervised machine learning, agreement between partitions is common...

Adjusting the adjusted Rand Index – A multinomial story

The Adjusted Rand Index (ARI) is arguably one of the most popular measur...

Adjusted Asymmetric Accuracy: A Well-Behaving External Cluster Validity Measure

There is no, nor will there ever be, single best clustering algorithm, b...

FastAMI – a Monte Carlo Approach to the Adjustment for Chance in Clustering Comparison Metrics

Clustering is at the very core of machine learning, and its applications...

Systematic Analysis of Cluster Similarity Indices: Towards Bias-free Cluster Validation

There are many cluster similarity indices used to evaluate clustering al...

On the Information Theoretic Distance Measures and Bidirectional Helmholtz Machines

By establishing a connection between bi-directional Helmholtz machines a...

Code Repositories


Code related to a paper about clustering comparisons available in Arxiv

view repo

Please sign up or login with your details

Forgot password? Click here to reset