Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning

by   Rahul Bera, et al.

Past research has proposed numerous hardware prefetching techniques, most of which rely on exploiting one specific type of program context information (e.g., program counter, cacheline address) to predict future memory accesses. These techniques either completely neglect a prefetcher's undesirable effects (e.g., memory bandwidth usage) on the overall system, or incorporate system-level feedback as an afterthought to a system-unaware prefetch algorithm. We show that prior prefetchers often lose their performance benefit over a wide range of workloads and system configurations due to their inherent inability to take multiple different types of program context and system-level feedback information into account while prefetching. In this paper, we make a case for designing a holistic prefetch algorithm that learns to prefetch using multiple different types of program context and system-level feedback information inherent to its design. To this end, we propose Pythia, which formulates the prefetcher as a reinforcement learning agent. For every demand request, Pythia observes multiple different types of program context information to make a prefetch decision. For every prefetch decision, Pythia receives a numerical reward that evaluates prefetch quality under the current memory bandwidth usage. Pythia uses this reward to reinforce the correlation between program context information and prefetch decision to generate highly accurate, timely, and system-aware prefetch requests in the future. Our extensive evaluations using simulation and hardware synthesis show that Pythia outperforms multiple state-of-the-art prefetchers over a wide range of workloads and system configurations, while incurring only 1.03 processor and no software changes in workloads. The source code of Pythia can be freely downloaded from


page 1

page 2

page 3

page 4


Sibyl: Adaptive and Extensible Data Placement in Hybrid Storage Systems Using Online Reinforcement Learning

Hybrid storage systems (HSS) use multiple different storage devices to p...

Online Application Guidance for Heterogeneous Memory Systems

Many high end and next generation computing systems to incorporated alte...

RLTF: Reinforcement Learning from Unit Test Feedback

The goal of program synthesis, or code generation, is to generate execut...

"Good Robot!": Efficient Reinforcement Learning for Multi-Step Visual Tasks via Reward Shaping

In order to learn effectively, robots must be able to extract the intang...

DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks

Data movement between the CPU and main memory is a first-order obstacle ...

Giving Semantics to Program-Counter Labels via Secure Effects

Type systems designed for information-flow control commonly use a progra...

A Generative Neural Network Framework for Automated Software Testing

Search Based Software Testing (SBST) is a popular automated testing tech...

Please sign up or login with your details

Forgot password? Click here to reset