Similarity Join and Similarity Self-Join Size Estimation in a Streaming Environment

by   Davood Rafiei, et al.

We study the problem of similarity self-join and similarity join size estimation in a streaming setting where the goal is to estimate, in one scan of the input and with sublinear space in the input size, the number of record pairs that have a similarity within a given threshold. The problem has many applications in data cleaning and query plan generation, where the cost of a similarity join may be estimated before actually doing the join. On unary input where two records either match or don't match, the problem becomes join and self-join size estimation for which one-pass algorithms are readily available. Our work addresses the problem for d-ary input, for d >= 1, where the degree of similarity can vary from 1 to d. We show that our proposed algorithm gives an accurate estimate and scales well with the input size. We provide error bounds and time and space costs, and conduct an extensive experimental evaluation of our algorithm, comparing its estimation accuracy to a few competitors, including some multi-pass algorithms. Our results show that given the same space, the proposed algorithm has an order of magnitude less error for a large range of similarity thresholds.


Similarity Join and Self-Join Size Estimation in a Streaming Environment

We study the problem of similarity self-join and similarity join size es...

Similarity Join Size Estimation using Locality Sensitive Hashing

Similarity joins are important operations with a broad range of applicat...

Preference-driven Similarity Join

Similarity join, which can find similar objects (e.g., products, names, ...

Improving Distributed Similarity Join in Metric Space with Error-bounded Sampling

Given two sets of objects, metric similarity join finds all similar pair...

Auto-FuzzyJoin: Auto-Program Fuzzy Similarity Joins Without Labeled Examples

Fuzzy similarity join is an important database operator widely used in p...

Efficient Taxonomic Similarity Joins with Adaptive Overlap Constraint

A similarity join aims to find all similar pairs between two collections...

GB-KMV: An Augmented KMV Sketch for Approximate Containment Similarity Search

In this paper, we study the problem of approximate containment similarit...

Please sign up or login with your details

Forgot password? Click here to reset