SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning

by   Immanuel Trummer, et al.

SkinnerDB is designed from the ground up for reliable join ordering. It maintains no data statistics and uses no cost or cardinality models. Instead, it uses reinforcement learning to learn optimal join orders on the fly, during the execution of the current query. To that purpose, we divide the execution of a query into many small time slices. Different join orders are tried in different time slices. We merge result tuples generated according to different join orders until a complete result is obtained. By measuring execution progress per time slice, we identify promising join orders as execution proceeds. Along with SkinnerDB, we introduce a new quality criterion for query execution strategies. We compare expected execution cost against execution cost for an optimal join order. SkinnerDB features multiple execution strategies that are optimized for that criterion. Some of them can be executed on top of existing database systems. For maximal performance, we introduce a customized execution engine, facilitating fast join order switching via specialized multi-way join algorithms and tuple representations. We experimentally compare SkinnerDB's performance against various baselines, including MonetDB, Postgres, and adaptive processing methods. We consider various benchmarks, including the join order benchmark and TPC-H variants with user-defined functions. Overall, the overheads of reliable join ordering are negligible compared to the performance impact of the occasional, catastrophic join order choice.


ADOPT: Adaptively Optimizing Attribute Orders for Worst-Case Optimal Join Algorithms via Reinforcement Learning

The performance of worst-case optimal join algorithms depends on the ord...

Deep Reinforcement Learning for Join Order Enumeration

Join order selection plays a significant role in query performance. Many...

Simpli-Squared: A Very Simple Yet Unexpectedly Powerful Join Ordering Algorithm Without Cardinality Estimates

The Join Order Benchmark (JOB) has become the de facto standard to asses...

Offset-value coding in database query processing

Recent work shows how offset-value coding speeds up database query execu...

Adaptive Geospatial Joins for Modern Hardware

Geospatial joins are a core building block of connected mobility applica...

Hiperfact: In-Memory High Performance Fact Processing – Rethinking the Rete Inference Algorithm

The Rete forward inference algorithm forms the basis for many rule engin...

Assessing the Impact of Execution Environment on Observation-Based Slicing

Program slicing reduces a program to a smaller version that retains a ch...

Please sign up or login with your details

Forgot password? Click here to reset