Incremental Techniques for Large-Scale Dynamic Query Processing

by   Iman Elghandour, et al.

Many applications from various disciplines are now required to analyze fast evolving big data in real time. Various approaches for incremental processing of queries have been proposed over the years. Traditional approaches rely on updating the results of a query when updates are streamed rather than re-computing these queries, and therefore, higher execution performance is expected. However, they do not perform well for large databases that are updated at high frequencies. Therefore, new algorithms and approaches have been proposed in the literature to address these challenges by, for instance, reducing the complexity of processing updates. Moreover, many of these algorithms are now leveraging distributed streaming platforms such as Spark Streaming and Flink. In this tutorial, we briefly discuss legacy approaches for incremental query processing, and then give an overview of the new challenges introduced due to processing big data streams. We then discuss in detail the recently proposed algorithms that address some of these challenges. We emphasize the characteristics and algorithmic analysis of various proposed approaches and conclude by discussing future research directions.


page 1

page 2

page 3

page 4


Geospatial Big Data Handling with High Performance Computing: Current Approaches and Future Directions

Geospatial big data plays a major role in the era of big data, as most d...

Lambda Learner: Fast Incremental Learning on Data Streams

One of the most well-established applications of machine learning is in ...

GAN-based Tabular Data Generator for Constructing Synopsis in Approximate Query Processing: Challenges and Solutions

In data-driven systems, data exploration is imperative for making real-t...

Online Aggregation based Approximate Query Processing: A Literature Survey

In the current world, OLAP (Online Analytical Processing) is used intens...

Efficient Multiple Incremental Computation for Kernel Ridge Regression with Bayesian Uncertainty Modeling

This study presents an efficient incremental/decremental approach for bi...

Search on Secondary Attributes in Geo-Distributed Systems

In the age of big data, more and more applications need to query and ana...

Practice of Streaming and Dynamic Graphs: Concepts, Models, Systems, and Parallelism

Graph processing has become an important part of various areas of comput...

Please sign up or login with your details

Forgot password? Click here to reset