Linear-Complexity Relaxed Word Mover's Distance with GPU Acceleration

by   Kubilay Atasu, et al.

The amount of unstructured text-based data is growing every day. Querying, clustering, and classifying this big data requires similarity computations across large sets of documents. Whereas low-complexity similarity metrics are available, attention has been shifting towards more complex methods that achieve a higher accuracy. In particular, the Word Mover's Distance (WMD) method proposed by Kusner et al. is a promising new approach, but its time complexity grows cubically with the number of unique words in the documents. The Relaxed Word Mover's Distance (RWMD) method, again proposed by Kusner et al., reduces the time complexity from qubic to quadratic and results in a limited loss in accuracy compared with WMD. Our work contributes a low-complexity implementation of the RWMD that reduces the average time complexity to linear when operating on large sets of documents. Our linear-complexity RWMD implementation, henceforth referred to as LC-RWMD, maps well onto GPUs and can be efficiently distributed across a cluster of GPUs. Our experiments on real-life datasets demonstrate 1) a performance improvement of two orders of magnitude with respect to our GPU-based distributed implementation of the quadratic RWMD, and 2) a performance improvement of three to four orders of magnitude with respect to our distributed WMD implementation that uses GPU-based RWMD for pruning.


page 1

page 2

page 3

page 4


Low-Complexity Data-Parallel Earth Mover's Distance Approximations

The Earth Mover's Distance (EMD) is a state-of-the art metric for compar...

Implementation Notes for the Soft Cosine Measure

The standard bag-of-words vector space model (VSM) is efficient, and ubi...

Parallel Graph Coloring Algorithms for Distributed GPU Environments

Graph coloring is often used in parallelizing scientific computations th...

SaberLDA: Sparsity-Aware Learning of Topic Models on GPUs

Latent Dirichlet Allocation (LDA) is a popular tool for analyzing discre...

Faster provable sieving algorithms for the Shortest Vector Problem and the Closest Vector Problem on lattices in ℓ_p norm

In this paper we give provable sieving algorithms for the Shortest Vecto...

Speeding up Word Mover's Distance and its variants via properties of distances between embeddings

The Word Mover's Distance (WMD) proposed in Kusner et al. [ICML,2015] is...

Please sign up or login with your details

Forgot password? Click here to reset