T^3OMVP: A Transformer-based Time and Team Reinforcement Learning Scheme for Observation-constrained Multi-Vehicle Pursuit in Urban Area

by   Zheng Yuan, et al.
Beijing University of Posts and Telecommunications

Smart Internet of Vehicles (IoVs) combined with Artificial Intelligence (AI) will contribute to vehicle decision-making in the Intelligent Transportation System (ITS). Multi-Vehicle Pursuit games (MVP), a multi-vehicle cooperative ability to capture mobile targets, is becoming a hot research topic gradually. Although there are some achievements in the field of MVP in the open space environment, the urban area brings complicated road structures and restricted moving spaces as challenges to the resolution of MVP games. We define an Observation-constrained MVP (OMVP) problem in this paper and propose a Transformer-based Time and Team Reinforcement Learning scheme (T^3OMVP) to address the problem. First, a new multi-vehicle pursuit model is constructed based on decentralized partially observed Markov decision processes (Dec-POMDP) to instantiate this problem. Second, by introducing and modifying the transformer-based observation sequence, QMIX is redefined to adapt to the complicated road structure, restricted moving spaces and constrained observations, so as to control vehicles to pursue the target combining the vehicle's observations. Third, a multi-intersection urban environment is built to verify the proposed scheme. Extensive experimental results demonstrate that the proposed T^3OMVP scheme achieves significant improvements relative to state-of-the-art QMIX approaches by 9.66 available at https://github.com/pipihaiziguai/T3OMVP.


page 2

page 5

page 8

page 9

page 13


Toward multi-target self-organizing pursuit in a partially observable Markov game

The multiple-target self-organizing pursuit (SOP) problem has wide appli...

A Multi-intersection Vehicular Cooperative Control based on End-Edge-Cloud Computing

Cooperative Intelligent Transportation Systems (C-ITS) will change the m...

Multi-robot Cooperative Pursuit via Potential Field-Enhanced Reinforcement Learning

It is of great challenge, though promising, to coordinate collective rob...

Decentralized Reinforcement Learning for Multi-Target Search and Detection by a Team of Drones

Targets search and detection encompasses a variety of decision problems ...

Semantic-level Decentralized Multi-Robot Decision-Making using Probabilistic Macro-Observations

Robust environment perception is essential for decision-making on robots...

An Efficient Algorithm for Multiple-Pursuer-Multiple-Evader Pursuit/Evasion Game

We present a method for pursuit/evasion that is highly efficient and and...

Please sign up or login with your details

Forgot password? Click here to reset