基于群体熵的机器人群体智能汇聚度量

doi:10.11959/j.issn.2096-6652.202213

智能科学与技术学报 ›› 2022, Vol. 4 ›› Issue (1): 65-74.doi: 10.11959/j.issn.2096-6652.202213

基于群体熵的机器人群体智能汇聚度量

冯埔¹, 吴文峻², 罗杰¹, 于鑫¹, 田雍恺¹

¹ 北京航空航天大学计算机学院，北京 100191
² 北京航空航天大学人工智能研究院，北京 100191

修回日期:2022-01-19 出版日期:2022-03-15 发布日期:2022-03-01
作者简介:冯埔（1995– ），男，北京航空航天大学计算机学院博士生，主要研究方向为多智能体强化学习
吴文峻（1973– ），男，博士，北京航空航天大学人工智能研究院教授、博士生导师，主要研究方向为群体智能、智能微服务、认知建模等
罗杰（1981– ），男，博士，北京航空航天大学计算机学院副教授，主要研究方向为知识图谱构、群体智能等
于鑫（1994– ），男，北京航空航天大学计算机学院博士生，主要研究方向为群体智能、强化学习
田雍恺（1997– ），男，北京航空航天大学计算机学院硕士生，主要研究方向为多智能体强化学习
基金资助:
科技创新2030—“新一代人工智能”重大项目(2018AAA0102300)

Emergence measurement of robot swarm intelligence based on swarm entropy

Pu FENG¹, Wenjun WU², Jie LUO¹, Xin YU¹, Yongkai TIAN¹

¹ School of Computer Science and Engineering, Beihang University, Beijing 100191, China
² Institute of Artificial Intelligence, Beihang University, Beijing 100191, China

Revised:2022-01-19 Online:2022-03-15 Published:2022-03-01
Supported by:
Science and Technology Innovation 2030 — “New Generation Artificial Intelligence” Major Project(2018AAA0102300)

摘要/Abstract

摘要：

群体行为往往能产生远超个体行为的价值和复杂度。为了在个体智能的基础上更有效地衍生出群体智能，需要基于群体熵来科学地衡量群体智能水平，并以群体熵为引导目标，推动群体智能的增强和演进。针对这个重要的科学问题，以无人小车群体为研究对象，提出基于参数共享和群体策略熵的多智能体soft Q learning算法，通过共享智能体的观测信息，并结合最大熵强化学习方法，实现探索型任务中群体策略的持续学习更新。同时，通过将群体熵定义为度量工具，刻画群体学习中熵变化模式，实现对群智汇聚过程的定量分析。

关键词: 群体熵, 群体智能, 深度强化学习

Abstract:

Swarm behavior can often produce value and complexity far beyond individual behavior.In order to more effectively derive swarm intelligence on the basis of individual intelligence, it is necessary to scientifically measure the level of swarm intelligence based on swarm entropy, and use swarm entropy as the guiding goal to promote the enhancement and evolution of swarm intelligence.Aiming at this important scientific problem, the unmanned car group as the research object was taken and a multi-agent soft Q learning method based on parameter sharing and group strategy entropy was proposed.Which by sharing the observation information of the agent, combined with the maximum entropy reinforcement learning method, to achieve continuous learning and updating of swarm strategies in exploratory tasks.At the same time, by defining swarm entropy as a measurement tool, characterizing the entropy change pattern in swarm learning, realizing the quantitative analysis of the gathering process of swarm intelligence.

Key words: swarm entropy, swarm intelligence, deep reinforcement learning

中图分类号:

TP181

冯埔, 吴文峻, 罗杰, 等. 基于群体熵的机器人群体智能汇聚度量[J]. 智能科学与技术学报, 2022, 4(1): 65-74.

Pu FENG, Wenjun WU, Jie LUO, et al. Emergence measurement of robot swarm intelligence based on swarm entropy[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 65-74.

图/表 10

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

参考文献 22

[1]	ZHU Y F , TANG X M . Overview of swarm intelligence[C]// Proceedings of 2010 International Conference on Computer Application and System Modeling (ICCASM 2010). Piscataway:IEEE Press, 2010.
[2]	REYNOLDS C W , . Flocks,herds and schools:a distributed behavioral model[C]// Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques. New York:ACM Press, 1987: 25-34.
[3]	VICSEK T , CZIRóK A , BEN-JACOB E ,et al. Novel type of phase transition in a system of self-driven particles[J]. Physical Review Letters, 1995,75(6): 1226-1229.
[4]	DEB K , PRATAP A , AGARWAL S ,et al. A fast and elitist multiobjective genetic algorithm:NSGA-II[J]. IEEE Transactions on Evolutionary Computation, 2002,6(2): 182-197.
[5]	DORIGO M , BLUM C . Ant colony optimization theory:a survey[J]. Theoretical Computer Science, 2005,344(2/3): 243-278.
[6]	KR?SE B J A . Learning from delayed rewards[J]. Robotics and Autonomous Systems, 1995,15(4): 233-235.
[7]	SCHULMAN J , WOLSKI F , DHARIWAL P ,et al. Proximal policy optimization algorithms[J]. arXiv preprint, 2017,arXiv:1707.06347.
[8]	FOERSTER J N , ASSAEL Y M , DE FREITAS N ,et al. Learning to communicate with deep multi-agent reinforcement learning[J]. Advances in Neural Information Processing Systems, 2016,29.
[9]	SHANNON C E . A mathematical theory of communication[J]. The Bell System Technical Journal, 1948,27(3): 379-423.
[10]	李习彬 . 熵-信息理论与系统工程方法论的有效性分析[J]. 系统工程理论与实践, 1994,14(2): 37-42.
	LI X B . Entropy-information theory and an analysis of the effectiveness of systems engineering’s methodology[J]. Systems Engineering Theory and Practice, 1994,14(2): 37-42.
[11]	KAUFFMAN S A . The origins of order:self-organization and selection in evolution[M]. New York: Oxford University Press, 1993.
[12]	CORNEEL C . Emergent collective motion from local interactions[D]. Ghent:Ghent University, 2017.
[13]	FRANCESCHINI V , GIBERTI C , ZHENG Z M . Characterization of the Lorentz attractor by unstable periodic orbits[J]. Nonlinearity, 1993,6(2): 251-258.
[14]	罗杰, 姜鑫, 郭炳晖 ,等. 群体智能系统的动力学模型与群体熵度量[J]. 中国科学:信息科学, 2022,52(1): 99-110.
	LUO J , JIANG X , GUO B H ,et al. Dynamic model and crowd entropy measurement of crowd intelligence system[J]. Scientia Sinica (Informationis), 2022,52(1): 99-110.
[15]	于鑫, 吴文峻, 罗杰 ,等. 面向群体共识机制的逆强化学习辨识方法[J]. 中国科学:技术科学, 2021.
	YU X , WU W J , LUO J ,et al. Inverse reinforcement learning identification method for group consensus mechanism[J]. Scientia Sinica (Technologica), 2021.
[16]	YU X , WU W J , FENG P ,et al. Swarm inverse reinforcement learning for biological systems[C]// Proceedings of 2021 IEEE International Conference on Bioinformatics and Biomedicine. Piscataway:IEEE Press, 2021: 274-279.
[17]	LOWE R , WU Y , TAMAR A ,et al. Multi-agent actor-critic for mixed cooperative-competitive environments[J]. arXiv preprint, 2017,arXiv:1706.02275.
[18]	SUNEHAG P , LEVER G , GRUSLYS A ,et al. Value-decomposition networks for cooperative multi-agent learning[J]. arXiv preprint, 2017,arXiv:1706.05296.
[19]	RASHID T , SAMVELYAN M , DE WITT C S ,et al. QMIX:monotonic value function factorisation for deep multi-agent reinforcement learning[J]. arXiv preprint, 2018,arXiv:1803.11485.
[20]	HAARNOJA T , TANG H , ABBEEL P ,et al. Reinforcement learning with deep energy-based policies[C]// Proceedings of the International Conference on Machine Learning.[S.l.:s.n.], 2017: 1352-1361.
[21]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Playing atari with deep reinforcement learning[J]. Computer Science, 2013.
[22]	RIPLEY B D , ROSENKRANTZ R D E T . Jaynes:papers on probability,statistics and statistical physics[J]. The Statistician, 1990,39(1): 90.

基于群体熵的机器人群体智能汇聚度量

Emergence measurement of robot swarm intelligence based on swarm entropy

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 22

相关文章 12

Metrics

推荐阅读 0

[1]	马帅, 傅启明, 陈建平, 冯帆, 陆悠, 李铮伟, 裘舒年. 基于双池DQN的HVAC无模型优化控制方法[J]. 智能科学与技术学报, 2022, 4(3): 426-444.
[2]	王雨倩, 丁嵘. 基于群体智能成果的路径规划程序自动生成系统[J]. 智能科学与技术学报, 2022, 4(2): 255-263.
[3]	刘家成, 张向文. 基于TD3的电动汽车复合电源能量管理策略研究[J]. 智能科学与技术学报, 2022, 4(2): 277-287.
[4]	孙宇祥, 彭益辉, 李斌, 周佳炜, 张鑫磊, 周献中. 智能博弈综述：游戏AI对作战推演的启示[J]. 智能科学与技术学报, 2022, 4(2): 157-173.
[5]	杨强, 童咏昕, 王晏晟, 范力欣, 王薇, 陈雷, 王魏, 康焱. 群体智能中的联邦学习算法综述[J]. 智能科学与技术学报, 2022, 4(1): 29-44.
[6]	项羽铭, 陈焜, 赵志峰, 李荣鹏, 张宏纲. 脑注意力机制启发的群体智能协同避障方法[J]. 智能科学与技术学报, 2022, 4(1): 84-96.
[7]	胡志强. 大数据智能指挥控制内在机理框架模型研究[J]. 智能科学与技术学报, 2021, 3(1): 101-109.
[8]	刘朝阳, 穆朝絮, 孙长银. 深度强化学习算法与应用研究现状综述[J]. 智能科学与技术学报, 2020, 2(4): 314-326.
[9]	李涛, 魏庆来. 基于深度强化学习的智能暖气温度控制系统[J]. 智能科学与技术学报, 2020, 2(4): 348-353.
[10]	王日中, 李慧平, 崔迪, 徐德民. 基于深度强化学习算法的自主式水下航行器深度控制[J]. 智能科学与技术学报, 2020, 2(4): 354-360.
[11]	傅汇乔, 唐开强, 邓归洲, 王鑫鹏, 陈春林. 基于深度强化学习的六足机器人运动规划[J]. 智能科学与技术学报, 2020, 2(4): 361-371.
[12]	沈宇,韩金朋,李灵犀,王飞跃. 游戏智能中的AI——从多角色博弈到平行博弈[J]. 智能科学与技术学报, 2020, 2(3): 205-213.