BCRLSP: An Offline Reinforcement Learning Framework for Sequential Targeted Promotion

by   Fanglin Chen, et al.

We utilize an offline reinforcement learning (RL) model for sequential targeted promotion in the presence of budget constraints in a real-world business environment. In our application, the mobile app aims to boost customer retention by sending cash bonuses to customers and control the costs of such cash bonuses during each time period. To achieve the multi-task goal, we propose the Budget Constrained Reinforcement Learning for Sequential Promotion (BCRLSP) framework to determine the value of cash bonuses to be sent to users. We first find out the target policy and the associated Q-values that maximizes the user retention rate using an RL model. A linear programming (LP) model is then added to satisfy the constraints of promotion costs. We solve the LP problem by maximizing the Q-values of actions learned from the RL model given the budget constraints. During deployment, we combine the offline RL model with the LP model to generate a robust policy under the budget constraints. Using both online and offline experiments, we demonstrate the efficacy of our approach by showing that BCRLSP achieves a higher long-term customer retention rate and a lower cost than various baselines. Taking advantage of the near real-time cost control method, the proposed framework can easily adapt to data with a noisy behavioral policy and/or meet flexible budget constraints.


Safe Offline Reinforcement Learning with Real-Time Budget Constraints

Aiming at promoting the safe real-world deployment of Reinforcement Lear...

Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning

We study the problem of safe offline reinforcement learning (RL), the go...

Revisiting the Linear-Programming Framework for Offline RL with General Function Approximation

Offline reinforcement learning (RL) concerns pursuing an optimal policy ...

Robust Reinforcement Learning using Offline Data

The goal of robust reinforcement learning (RL) is to learn a policy that...

Optimal Bidding Strategy without Exploration in Real-time Bidding

Maximizing utility with a budget constraint is the primary goal for adve...

Timing is Everything: Learning to Act Selectively with Costly Actions and Budgetary Constraints

Many real-world settings involve costs for performing actions; transacti...

Benchmarking Offline Reinforcement Learning Algorithms for E-Commerce Order Fraud Evaluation

Amazon and other e-commerce sites must employ mechanisms to protect thei...

Please sign up or login with your details

Forgot password? Click here to reset