Statistically Significant Pattern Mining with Ordinal Utility

by   Thien Q. Tran, et al.

Statistically significant patterns mining (SSPM) is an essential and challenging data mining task in the field of knowledge discovery in databases (KDD), in which each pattern is evaluated via a hypothesis test. Our study aims to introduce a preference relation into patterns and to discover the most preferred patterns under the constraint of statistical significance, which has never been considered in existing SSPM problems. We propose an iterative multiple testing procedure that can alternately reject a hypothesis and safely ignore the hypotheses that are less useful than the rejected hypothesis. One advantage of filtering out patterns with low utility is that it avoids consumption of the significance budget by rejection of useless (that is, uninteresting) patterns. This allows the significance budget to be focused on useful patterns, leading to more useful discoveries. We show that the proposed method can control the familywise error rate (FWER) under certain assumptions, that can be satisfied by a realistic problem class in SSPM.We also show that the proposed method always discovers a set of patterns that is at least equally or more useful than those discovered using the standard Tarone-Bonferroni method SSPM.Finally, we conducted several experiments with both synthetic and real-world data to evaluate the performance of our method. As a result, in the experiments with real-world datasets, the proposed method discovered a larger number of more useful patterns than the existing method for all five conducted tasks.


page 1

page 2

page 3

page 4


Statistically Significant Discriminative Patterns Searching

Discriminative pattern mining is an essential task of data mining. This ...

Searching for significant patterns in stratified data

Significant pattern mining, the problem of finding itemsets that are sig...

Selective Inference Approach for Statistically Sound Predictive Pattern Mining

Discovering statistically significant patterns from databases is an impo...

Flexible Pattern Discovery and Analysis

Based on the analysis of the proportion of utility in the supporting tra...

Beyond Frequency: Utility Mining with Varied Item-Specific Minimum Utility

Utility-oriented mining which integrates utility theory and data mining ...

Significant Subgraph Mining with Multiple Testing Correction

The problem of finding itemsets that are statistically significantly enr...

Label-Descriptive Patterns and their Application to Characterizing Classification Errors

State-of-the-art deep learning methods achieve human-like performance on...

Please sign up or login with your details

Forgot password? Click here to reset