Memory-Efficient Group-by Aggregates over Multi-Way Joins

Aggregate computation in relational databases has long been done using the standard unary aggregation and binary join operators. These implement the classical model of computing joins between relations two at a time, materializing the intermediate results, and then proceeding to do either sort-based or hash-based aggregate computation to derive the final result. This approach, however, can be dramatically sub-optimal in the case of low-selectivity joins, and often ends up generating large intermediate results even if the relations involved as well as the final result sets themselves are quite small. Moreover, many of the computed intermediate results may never be part of the final result. In this paper we propose a novel aggregate query processing technique that leverages graph data structures towards efficiently answering aggregate queries over joins, without materializing intermediate join results. We wrap this technique inside a multi-way composite database operator called JOIN-AGG that combines Join and Aggregation. We provide a general framework for executing aggregation queries over arbitrary acyclic joins, involving any number of group-by attributes from any relation. We also present a thorough experimental evaluation on both real world and synthetic datasets. Our experiments show that our operators can achieve orders of magnitude lower query times and memory requirements compared to the traditional approach, even when implemented outside of the database system.


page 1

page 2

page 3

page 4


SieveJoin: Boosting Multi-Way Joins with Reusable Bloom Filters

Improving data systems' performance for join operations has long been an...

Conjunctive Queries with Theta Joins Under Updates

Modern application domains such as Composite Event Recognition (CER) and...

Aggregation Consistency Errors in Semantic Layers and How to Avoid Them

Analysts often struggle with analyzing data from multiple tables in a da...

Hiperfact: In-Memory High Performance Fact Processing – Rethinking the Rete Inference Algorithm

The Rete forward inference algorithm forms the basis for many rule engin...

Generalized Lineage-Aware Temporal Windows: Supporting Outer and Anti Joins in Temporal-Probabilistic Databases

The result of a temporal-probabilistic (TP) join with negation includes,...

Sort-based grouping and aggregation

Database query processing requires algorithms for duplicate removal, gro...

APRIL: Approximating Polygons as Raster Interval Lists

The spatial intersection join an important spatial query operation, due ...

Please sign up or login with your details

Forgot password? Click here to reset