A Network Perspective on Stratification of Multi-Label Data

by   Piotr Szymański, et al.

In the recent years, we have witnessed the development of multi-label classification methods which utilize the structure of the label space in a divide and conquer approach to improve classification performance and allow large data sets to be classified efficiently. Yet most of the available data sets have been provided in train/test splits that did not account for maintaining a distribution of higher-order relationships between labels among splits or folds. We present a new approach to stratifying multi-label data for classification purposes based on the iterative stratification approach proposed by Sechidis et. al. in an ECML PKDD 2011 paper. Our method extends the iterative approach to take into account second-order relationships between labels. Obtained results are evaluated using statistical properties of obtained strata as presented by Sechidis. We also propose new statistical measures relevant to second-order quality: label pairs distribution, the percentage of label pairs without positive evidence in folds and label pair - fold pairs that have no positive evidence for the label pair. We verify the impact of new methods on classification performance of Binary Relevance, Label Powerset and a fast greedy community detection based label space partitioning classifier. Random Forests serve as base classifiers. We check the variation of the number of communities obtained per fold, and the stability of their modularity score. Second-Order Iterative Stratification is compared to standard k-fold, label set, and iterative stratification. The proposed approach lowers the variance of classification quality, improves label pair oriented measures and example distribution while maintaining a competitive quality in label-oriented measures. We also witness an increase in stability of network characteristics.


page 1

page 2

page 3

page 4


EvoSplit: An evolutionary approach to split a multi-label data set into disjoint subsets

This paper presents a new evolutionary approach, EvoSplit, for the distr...

Understanding Label Bias in Single Positive Multi-Label Learning

Annotating data for multi-label classification is prohibitively expensiv...

How is a data-driven approach better than random choice in label space division for multi-label classification?

We propose using five data-driven community detection approaches from so...

Applying an Ensemble Learning Method for Improving Multi-label Classification Performance

In recent years, multi-label classification problem has become a controv...

Weighting Scheme for a Pairwise Multi-label Classifier Based on the Fuzzy Confusion Matrix

In this work we addressed the issue of applying a stochastic classifier ...

Nearest Labelset Using Double Distances for Multi-label Classification

Multi-label classification is a type of supervised learning where an ins...

Bottom-up Hierarchical Classification Using Confusion-based Logit Compression

In this work, we propose a method to efficiently compute label posterior...

Please sign up or login with your details

Forgot password? Click here to reset