Tempura: A General Cost Based Optimizer Framework for Incremental Data Processing (Extended Version)

by   Zuozhi Wang, et al.
University of California, Irvine
Alibaba Group

Incremental processing is widely-adopted in many applications, ranging from incremental view maintenance, stream computing, to recently emerging progressive data warehouse and intermittent query processing. Despite many algorithms developed on this topic, none of them can produce an incremental plan that always achieves the best performance, since the optimal plan is data dependent. In this paper, we develop a novel cost-based optimizer framework, called Tempura, for optimizing incremental data processing. We propose an incremental query planning model called TIP based on the concept of time-varying relations, which can formally model incremental processing in its most general form. We give a full specification of Tempura, which can not only unify various existing techniques to generate an optimal incremental plan, but also allow the developer to add their rewrite rules. We study how to explore the plan space and search for an optimal incremental plan. We conduct a thorough experimental evaluation of Tempura in various incremental processing scenarios to show its effectiveness and efficiency.


page 14

page 16


A Survey on Advancing the DBMS Query Optimizer: Cardinality Estimation, Cost Model, and Plan Enumeration

Query optimizer is at the heart of the database systems. Cost-based opti...

DBSP: Automatic Incremental View Maintenance for Rich Query Languages

Incremental view maintenance has been for a long time a central problem ...

Incremental View Maintenance for Property Graph Queries

This paper discusses the challenges of incremental view maintenance for ...

Optimizing Subgraph Queries by Combining Binary and Worst-Case Optimal Joins

We study the problem of optimizing subgraph queries using the new worst-...

Understanding and Extending Incremental Determinization for 2QBF

Incremental determinization is a recently proposed algorithm for solving...

Incremental Noising and its Fractal Behavior

This manuscript is about further elucidating the concept of noising. The...

QueryER: A Framework for Fast Analysis-Aware Deduplication over Dirty Data

In this work, we explore the problem of correctly and efficiently answer...

Please sign up or login with your details

Forgot password? Click here to reset