Embedding Contextual Information through Reward Shaping in Multi-Agent Learning: A Case Study from Google Football

by   Chaoyi Gu, et al.

Artificial Intelligence has been used to help human complete difficult tasks in complicated environments by providing optimized strategies for decision-making or replacing the manual labour. In environments including multiple agents, such as football, the most common methods to train agents are Imitation Learning and Multi-Agent Reinforcement Learning (MARL). However, the agents trained by Imitation Learning cannot outperform the expert demonstrator, which makes humans hardly get new insights from the learnt policy. Besides, MARL is prone to the credit assignment problem. In environments with sparse reward signal, this method can be inefficient. The objective of our research is to create a novel reward shaping method by embedding contextual information in reward function to solve the aforementioned challenges. We demonstrate this in the Google Research Football (GRF) environment. We quantify the contextual information extracted from game state observation and use this quantification together with original sparse reward to create the shaped reward. The experiment results in the GRF environment prove that our reward shaping method is a useful addition to state-of-the-art MARL algorithms for training agents in environments with sparse reward signal.


page 1

page 3

page 6


Reinforced Imitation in Heterogeneous Action Space

Imitation learning is an effective alternative approach to learn a polic...

Multi-Agent Generative Adversarial Imitation Learning

Imitation learning algorithms can be used to learn a policy from expert ...

Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Competitive Games

Training agents in multi-agent competitive games presents significant ch...

Health-Informed Policy Gradients for Multi-Agent Reinforcement Learning

This paper proposes a definition of system health in the context of mult...

Accelerating Training in Pommerman with Imitation and Reinforcement Learning

The Pommerman simulation was recently developed to mimic the classic Jap...

Selective imitation on the basis of reward function similarity

Imitation is a key component of human social behavior, and is widely use...

Please sign up or login with your details

Forgot password? Click here to reset