Tactical Reward Shaping: Bypassing Reinforcement Learning with Strategy-Based Goals

by   Yizheng Zhang, et al.

Deep Reinforcement Learning (DRL) has shown its promising capabilities to learn optimal policies directly from trial and error. However, learning can be hindered if the goal of the learning, defined by the reward function, is "not optimal". We demonstrate that by setting the goal/target of competition in a counter-intuitive but intelligent way, instead of heuristically trying solutions through many hours the DRL simulation can quickly converge into a winning strategy. The ICRA-DJI RoboMaster AI Challenge is a game of cooperation and competition between robots in a partially observable environment, quite similar to the Counter-Strike game. Unlike the traditional approach to games, where the reward is given at winning the match or hitting the enemy, our DRL algorithm rewards our robots when in a geometric-strategic advantage, which implicitly increases the winning chances. Furthermore, we use Deep Q Learning (DQL) to generate multi-agent paths for moving, which improves the cooperation between two robots by avoiding the collision. Finally, we implement a variant A* algorithm with the same implicit geometric goal as DQL and compare results. We conclude that a well-set goal can put in question the need for learning algorithms, with geometric-based searches outperforming DQL in many orders of magnitude.


page 2

page 5


Directionality Reinforcement Learning to Operate Multi-Agent System without Communication

This paper establishes directionality reinforcement learning (DRL) techn...

A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learning

Deep Reinforcement Learning (DRL) is a promising approach for teaching r...

A Memory Efficient Deep Reinforcement Learning Approach For Snake Game Autonomous Agents

To perform well, Deep Reinforcement Learning (DRL) methods require signi...

Decentralized Multi-Agent Reinforcement Learning with Global State Prediction

Deep reinforcement learning (DRL) has seen remarkable success in the con...

On the Generalization of Deep Reinforcement Learning Methods in the Problem of Local Navigation

In this paper, we study the application of DRL algorithms in the context...

Negative Update Intervals in Deep Multi-Agent Reinforcement Learning

In Multi-Agent Reinforcement Learning, independent cooperative learners ...

Proficiency Aware Multi-Agent Actor-Critic for Mixed Aerial and Ground Robot Teaming

Mixed Cooperation and competition are the actual scenarios of deploying ...

Please sign up or login with your details

Forgot password? Click here to reset