A Policy Gradient Algorithm to Alleviate the Multi-Agent Value Overestimation Problem in Complex Environments

doi:10.3390/s23239520

CIOMP OpenIR

	A Policy Gradient Algorithm to Alleviate the Multi-Agent Value Overestimation Problem in Complex Environments
	Y. Yang, J. Li, J. Hou, Y. Wang and H. Zhao
	2023
发表期刊	Sensors
ISSN	14248220
卷号	23 期号:23
摘要	Multi-agent reinforcement learning excels at addressing group intelligent decision-making problems involving sequential decision-making. In particular, in complex, high-dimensional state and action spaces, it imposes higher demands on the reliability, stability, and adaptability of decision algorithms. The reinforcement learning algorithm based on the multi-agent deep strategy gradient incorporates a function approximation method using discriminant networks. However, this can lead to estimation errors when agents evaluate action values, thereby reducing model reliability and stability and resulting in challenging convergence. With the increasing complexity of the environment, there is a decline in the quality of experience collected by the experience playback pool, resulting in low efficiency of the sampling stage and difficulties in algorithm convergence. To address these challenges, we propose an innovative approach called the empirical clustering layer-based multi-agent dual dueling policy gradient (ECL-MAD3PG) algorithm. Experimental results demonstrate that our ECL-MAD3PG algorithm outperforms other methods in various complex environments, demonstrating a remarkable 9.1% improvement in mission completion compared to MADDPG within the context of complex UAV cooperative combat scenarios. © 2023 by the authors.
DOI	10.3390/s23239520
URL	查看原文
收录类别	sci ; ei
引用统计
文献类型	期刊论文
条目标识符	http://ir.ciomp.ac.cn/handle/181722/68091
专题	中国科学院长春光学精密机械与物理研究所
推荐引用方式 GB/T 7714	Y. Yang, J. Li, J. Hou, Y. Wang and H. Zhao. A Policy Gradient Algorithm to Alleviate the Multi-Agent Value Overestimation Problem in Complex Environments[J]. Sensors,2023,23(23).
APA	Y. Yang, J. Li, J. Hou, Y. Wang and H. Zhao.(2023).A Policy Gradient Algorithm to Alleviate the Multi-Agent Value Overestimation Problem in Complex Environments.Sensors,23(23).
MLA	Y. Yang, J. Li, J. Hou, Y. Wang and H. Zhao."A Policy Gradient Algorithm to Alleviate the Multi-Agent Value Overestimation Problem in Complex Environments".Sensors 23.23(2023).

条目包含的文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
A Policy Gradient Al（3316KB）	期刊论文	出版稿	开放获取	CC BY-NC-SA	浏览请求全文