异构多智能体系统的输出同步：一个基于数据的强化学习方法

doi:10.11959/j.issn.2096-6652.202042

Abstract

Abstract:

The output synchronization of heterogeneous multi-agent system was studied by reinforcement learning.According to the topology of multi-agent system, the performance index and value function with neighbor control input were defined.To overcome the disadvantage of existing control methods that require system model, a reinforcement learning algorithm based on system data was proposed.Hence, the output synchronization controller can also be applied when the system model was unknown.In addition, by adjusting the weight matrix in value function, the control cost of each agent can be reduced.Finally, a simulation example was given to illustrate the effectiveness of proposed method and the superiority of defined value function.

Key words: multi-agent system, reinforcement learning, output synchronization, based on data

CLC Number:

TP273，TP301

Yingying LIU,Zhanshan WANG. Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data[J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 394-400.

Figures/Tables 6

References 20

[1]	LI X W , SOH Y C , XIE L H ,et al. Cooperative output regulation of heterogeneous linear multi-agent networks via H_∞performance allocation[J]. IEEE Transactions on Automatic Control, 2019,64(2): 683-696.
[2]	孙长银, 穆朝絮 . 多智能体深度强化学习的若干关键科学问题[J]. 自动化学报, 2020,7(46): 1301-1312.
	SUN C Y , MU C X . Important scientific problems of multi-agent deep reinforcement learning[J]. Acta Automatica Sinica, 2020,7(46): 1301-1312.
[3]	裴国旭, 杜晓明, 薛昭 ,等. 多智能体系统在军事仿真领域的应用现状[J]. 飞航导弹, 2017(2): 46-49,73.
	PEI G X , DU X M , XUE Z ,et al. Application status of multi-agent system in military simulation field[J]. Aerodynamic Missile Journal, 2017(2): 46-49,73.
[4]	ABDELKADER A , ABDELHAMID T . Distributed output regulation of heterogeneous linear multi-agent systems with communication constraints[J]. Automatica, 2018,91: 152-158.
[5]	史乐, 李辉, 原江波 . 基于消息通信的多智能体系统的应用[J]. 计算机应用, 2008,28(2): 531-534.
	SHI L , LI H , YUAN J B . Multi-agent system based on the message communicaiton[J]. Journal of Computer Applications, 2008,28(2): 531-534.
[6]	QIN J H , MA Q C , SHI Y ,et al. Recent advances in consensus of multi-agent systems:a brief survey[J]. IEEE Transactions on Industrial Electronics, 2017,64(6): 4972-4983.
[7]	ZHANG H G , JIANG H , LUO Y H ,et al. Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method[J]. IEEE Transactions on Industrial Electronics, 2017,64(5): 4091-4100.
[8]	WANG X , YANG G H . Fault-tolerant consensus tracking control for linear multi-agent systems under switching directed network[J]. IEEE Transactions on Cybernetics, 2020,50(5): 1921-1930.
[9]	ZHANG J C , ZHU F L . Observer-based output consensus of a class of heterogeneous multi-agent systems with unmatched disturbances[J]. Communications in Nonlinear Science and Numerical Simulation, 2018,56: 240-251.
[10]	LIU L . Adaptive cooperative output regulation for a class of nonlinear multi-agent systems[J]. IEEE Transactions on Automatic Control, 2015,60(6): 1677-1682.
[11]	MA Q , QIN J , ZHENG W X ,et al. Output group synchronization for networks of heterogeneous linear systems under internal model principle[J]. IEEE Transactions on Circuits and Systems I:Regular Papers, 2018,65(5): 1684-1695.
[12]	YAN Y M , HUANG J . Cooperative output regulation of discrete-time linear time-delay multi-agent systems[J]. IET Control Theory ＆ Applications, 2016,10(16): 2019-2026.
[13]	王飞跃, 曹东璞, 魏庆来 . 强化学习:迈向知行合一的智能机制与算法[J]. 智能科学与技术学报, 2020,2(2): 101-106.
	WANG F Y , CAO D P , WEI Q L . Reinforcement learning:toward actionknowledge merged intelligent mechanisms and algorithms[J]. Chinese Journal of Intelligent Science and Technology, 2020,2(2): 101-106.
[14]	LEWIS F L , VRABIE D . Reinforcement learning and adaptive dynamic programming for feedback control[J]. IEEE Circuits and Systems Magazine, 2009,9(3): 32-50.
[15]	LIU Y Y , WANG Z S , SHI Z . H_∞ tracking control for linear discrete-time systems via reinforcement learning[J]. International Journal of Robust and Nonlinear Control, 2020,30(1): 282-301.
[16]	MODARES H , NAGESHRAO S P , LOPES G A D ,et al. Optimal model-free output synchronization of heterogeneous systems using off-policy reinforcement learning[J]. Automatica, 2016,71: 334-341.
[17]	MODARES H , LEWIS F L , KANG W ,et al. Optimal synchronization of heterogeneous nonlinear systems with unknown dynamics[J]. IEEE Transactions on Automatic Control, 2018,63(1): 117-131.
[18]	KIUMARSI B , LEWIS F L , MODARES H ,et al. Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics[J]. Automatica, 2014,50: 1167-1175.
[19]	KIUMARSI B , LEWIS F L . Output synchronization of heterogeneous discrete-time systems:A model-free optimal approach[J]. Automatica, 2017,84: 86-94.
[20]	YANG Y L , MODARES H , WUNSCH D C ,et al. Leader-follower output synchronization of linear heterogeneous systems with active leader using reinforcement learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018,29(6): 2139-2153.

Metrics

Recommended 0

No Suggested Reading articles found!

Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 20

Related Articles 15

Metrics

Recommended 0

[1]	Zhou YU, Jing BI, Haitao YUAN. A path planning method for complex naval battle field based on an improved DQN algorithm [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 418-425.
[2]	Shuai MA, Qiming FU, Jianping CHEN, Fan FENG, You LU, Zhengwei LI, Shunian QIU. HVAC model-free optimal control method based on double-pools DQN [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 426-444.
[3]	Yuxiang SUN, Yihui PENG, Bin LI, Jiawei ZHOU, Xinlei ZHANG, Xianzhong ZHOU. Overview of intelligent game:enlightenment of game AI to combat deduction [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(2): 157-173.
[4]	De XU, Fangbo QIN. Research development on automated robotic peg-in-hole assembly [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(2): 200-211.
[5]	Jiacheng LIU, Xiangwen ZHANG. TD3-based energy management strategy for hybrid energy storage system of electric vehicle [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(2): 277-287.
[6]	Pu FENG, Wenjun WU, Jie LUO, Xin YU, Yongkai TIAN. Emergence measurement of robot swarm intelligence based on swarm entropy [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 65-74.
[7]	Han WANG, Yang YU, Yuan JIANG. A cooperative multi-agent reinforcement learning algorithm based on dynamic self-selection parameters sharing [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 75-83.
[8]	Lina XIA, Qing LI, Ruizhuo SONG, Zihan WANG, Zhen XU. Synchronization control of unknown heterogeneous multi-agent system via model-free adaptive dynamic programming [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 444-448.
[9]	Zhiqiang HU. The framework model on internal mechanism of big data intelligent command and control [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(1): 101-109.
[10]	Zhaoyang LIU, Chaoxu MU, Changyin SUN. An overview on algorithms and applications of deep reinforcement learning [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 314-326.
[11]	Jinna LI, Weiran CHENG. An overview of optimal consensus for data driven multi-agent system based on reinforcement learning [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 327-340.
[12]	Qing-Shan JIA, Jingxian TANG, Junjie WU, Xiao HU, Yiting LIN, Heng XIA. Reinforcement learning for green and reliable data center [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 341-347.
[13]	Tao LI, Qinglai WEI. Intelligent heating temperature control system based on deep reinforcement learning [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 348-353.
[14]	Rizhong WANG, Huiping LI, Di CUI, Demin XU. Depth control of autonomous underwater vehicle using deep reinforcement learning [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 354-360.
[15]	Huiqiao FU, Kaiqiang TANG, Guizhou DENG, Xinpeng WANG, Chunlin CHEN. Motion planning for hexapod robot using deep reinforcement learning [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 361-371.