Autonomous Tracking Using a Swarm of UAVs: A Constrained Multi-Agent Reinforcement Learning Approach

Yu Jia Chen, Deng Kai Chang, Cheng Zhang

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


In this paper, we aim to design an autonomous tracking system for a swarm of unmanned aerial vehicles (UAVs) to localize a radio frequency (RF) mobile target. In the system, UAVs equipped with omnidirectional received signal strength (RSS) sensors can cooperatively search the target with a specified tracking accuracy. To achieve fast localization and tracking in the highly dynamic channel environment (e.g., time-varying transmit power and intermittent signal), we formulate a flight decision problem as a constrained Markov decision process (CMDP) with the main objective of avoiding redundant UAV flight path. Then, we propose an enhanced multi-agent reinforcement learning to coordinate multiple UAVs performing real-time target tracking. The core of the proposed scheme is a feedback control system that takes into account the uncertainty of the channel estimate. We prove that the proposed algorithm can converge to the optimal decision. Our simulation results show that the proposed scheme outperforms standard Q-learning and multi-agent Q-learning algorithms in terms of searching time and successful localization probability.

Original languageEnglish
Article number9195795
Pages (from-to)13702-13717
Number of pages16
JournalIEEE Transactions on Vehicular Technology
Issue number11
StatePublished - Nov 2020


  • Multi-agent reinforcement learning
  • constrained Markov decision process
  • localization and tracking
  • unmanned aerial vehicles (UAVs)


Dive into the research topics of 'Autonomous Tracking Using a Swarm of UAVs: A Constrained Multi-Agent Reinforcement Learning Approach'. Together they form a unique fingerprint.

Cite this