Learning Temporal Point Processes via Reinforcement Learning

by   Shuang Li, et al.

Social goods, such as healthcare, smart city, and information networks, often produce ordered event data in continuous time. The generative processes of these event data can be very complex, requiring flexible models to capture their dynamics. Temporal point processes offer an elegant framework for modeling event data without discretizing the time. However, the existing maximum-likelihood-estimation (MLE) learning paradigm requires hand-crafting the intensity function beforehand and cannot directly monitor the goodness-of-fit of the estimated model in the process of training. To alleviate the risk of model-misspecification in MLE, we propose to generate samples from the generative model and monitor the quality of the samples in the process of training until the samples and the real data are indistinguishable. We take inspiration from reinforcement learning (RL) and treat the generation of each event as the action taken by a stochastic policy. We parameterize the policy as a flexible recurrent neural network and gradually improve the policy to mimic the observed event distribution. Since the reward function is unknown in this setting, we uncover an analytic and nonparametric form of the reward function using an inverse reinforcement learning formulation. This new RL framework allows us to derive an efficient policy gradient algorithm for learning flexible point process models, and we show that it performs well in both synthetic and real data.


page 1

page 2

page 3

page 4


Reinforcement Learning of Spatio-Temporal Point Processes

Spatio-temporal event data is ubiquitous in various applications, such a...

Reinforcement Learning with Policy Mixture Model for Temporal Point Processes Clustering

Temporal point process is an expressive tool for modeling event sequence...

Reinforcement Logic Rule Learning for Temporal Point Processes

We propose a framework that can incrementally expand the explanatory tem...

Deep Reinforcement Learning of Marked Temporal Point Processes

In a wide variety of applications, humans interact with a complex enviro...

Reinforcement Learning for Topic Models

We apply reinforcement learning techniques to topic modeling by replacin...

Conditional Generative Modeling is All You Need for Marked Temporal Point Processes

Recent advancements in generative modeling have made it possible to gene...

Connecting the Dots Between MLE and RL for Sequence Generation

Sequence generation models such as recurrent networks can be trained wit...

Please sign up or login with your details

Forgot password? Click here to reset