物联网数据收集中无人机路径智能规划

doi:10.11959/j.issn.1000-436x.2021036

通信学报 ›› 2021, Vol. 42 ›› Issue (2): 124-133.doi: 10.11959/j.issn.1000-436x.2021036

物联网数据收集中无人机路径智能规划

付澍¹^,², 杨祥月¹, 张海君³, 陈晨¹, 喻鹏⁴, 简鑫¹, 刘敏¹

¹ 重庆大学微电子与通信工程学院，重庆 400030
² 重庆大学信息物理社会可信服务计算教育部重点实验室，重庆 400030
³ 北京科技大学计算机与通信工程学院，北京 100083
⁴ 北京邮电大学网络与交换技术国家重点实验室，北京 100876

修回日期:2020-11-20 出版日期:2021-02-25 发布日期:2021-02-01
作者简介:付澍（1985- ），男，贵州贵阳人，博士，重庆大学副教授、硕士生导师，主要研究方向为星地通信、NOMA、物联网、网络一体化等。
杨祥月（1995- ），女，贵州毕节人，重庆大学硕士生，主要研究方向为机器学习、无人机数据收集、路径规划等。
张海君（1986- ），男，辽宁朝阳人，博士，北京科技大学教授、博士生导师，主要研究方向为6G移动通信、人工智能与无线网络等。
陈晨（1988- ），男，江苏淮安人，博士，重庆大学研究员、博士生导师，主要研究方向为面向6G和物联网的无线光通信技术。
喻鹏（1986− ），男，湖北随州人，博士，北京邮电大学副教授、博士生导师，主要研究方向为5G/6G网络智能管控。
简鑫（1987- ），男，四川自贡人，博士，重庆大学副教授、硕士生导师，主要研究方向为应用统计数学、物联网无线接入技术及应用、分布式算法等。
刘敏（1975- ），女，四川大竹人，重庆大学教授、博士生导师，主要研究方向为光通信与光网络。
基金资助:
国家自然科学基金资助项目(61701054);中央高校基本科研业务费专项资金资助项目(2020CDJQY-A001);中央高校基本科研业务费专项资金资助项目(2020CDJGFWDZ014)

UAV path intelligent planning in IoT data collection

Shu FU¹^,², Xiangyue YANG¹, Haijun ZHANG³, Chen CHEN¹, Peng YU⁴, Xin JIAN¹, Min LIU¹

¹ School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400030, China
² Key Laboratory of Dependable Service Computing in Cyber Physical Society of Ministry of Education, Chongqing University, Chongqing 400030, China
³ School of Computer ＆Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China
⁴ State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China

Revised:2020-11-20 Online:2021-02-25 Published:2021-02-01
Supported by:
The National Natural Science Foundation of China(61701054);The Fundamental Research Funds for the Central Universities(2020CDJQY-A001);The Fundamental Research Funds for the Central Universities(2020CDJGFWDZ014)

摘要/Abstract

摘要：

为解决无人机在数据收集过程中的路径规划问题，将其分为全局路径规划和局部路径规划。针对全局路径规划，将其建模为一个定向问题，定向问题是背包问题和旅行商问题2种经典优化问题的组合。采用指针网络深度学习对该模型进行求解，并在无人机能量约束下得到其服务节点集合及服务顺序。针对局部路径规划，基于无人机接收到节点的参考信号强度，通过深度Q网络学习对无人机局部飞行路径进行规划，使无人机逼近节点位置并服务各节点。仿真结果表明，所提方案能够在无人机能量约束下有效提升其数据收集的收益。

关键词: 无人机, 数据收集, 路径规划, 指针网络, 深度Q网络

Abstract:

To solve the problem of path planning of UAV data collection, it was generally be divided into global path planning and local path planning.For global path planning, it was modeled as an orientation problem, which was a combination of two classical optimization problems, the knapsack problem and the traveling salesman problem.The pointer network of deep learning was used to solve the model to obtain the service node set and service order under the energy constraint of the UAV.In terms of the local path planning, the reference signal strength (RSS) of the sensor node received by UAV was employed to learn the local flight path of UAV by deep Q network, which enabled the UAV to approach and serve the nodes.Simulation results show that the proposed scheme can effectively improve the revenue of UAV data collection under the energy constraint of UAV.

Key words: unmanned aerial vehicle, data collection, path planning, pointer network, deep Q network

中图分类号:

TN92

付澍, 杨祥月, 张海君, 陈晨, 喻鹏, 简鑫, 刘敏. 物联网数据收集中无人机路径智能规划[J]. 通信学报, 2021, 42(2): 124-133.

Shu FU, Xiangyue YANG, Haijun ZHANG, Chen CHEN, Peng YU, Xin JIAN, Min LIU. UAV path intelligent planning in IoT data collection[J]. Journal on Communications, 2021, 42(2): 124-133.

图/表 15

图1

图2

图3

图4

图5

表1

表2

表3

图6

图7

表4

图8

图9

图10

表5

参考文献 40

[1]	LI C , ZHANG H , HAO B ,et al. A survey on routing protocols for large-scale wireless sensor networks[J]. Sensors, 2011,11(12): 3498-3526.
[2]	ZENG Y , ZHANG R , LIM T J . Wireless communications with unmanned aerial vehicles:opportunities and challenges[J]. IEEE Communications Magazine, 2016,54(5): 36-42.
[3]	WANG C , MA F , YAN J ,et al. Efficient aerial data collection with UAV in large-scale wireless sensor networks[J]. International Journal of Distributed Sensor Networks, 2015,2015: 1-19.
[4]	UTKARSHA S.P , RAJIV K G . Clustering and compressive data gpthering in wireless sensor network[J]. Wireless Personal Communications, 2019,109(2): 1-21.
[5]	JIE G , TSUNG-HUI C , CHAO S ,et al. Flight time minimization of UAV for data collection over wireless sensor networks[J]. IEEE Journal on Selected Areas in Communications, 2018,PP(99): 1.
[6]	ZAIN A A , SUHAIB M , MUHAMMAD A . UAV based data gpthering in wireless sensor networks[J]. Wireless Personal Communications, 2019,106(4): 1-11.
[7]	DUAN H , LI P . UAV path planning[M]. Berlin: Springer, 2014.
[8]	ARVANITAKI A , PAPPAS N . Modeling of a UAV-based data collection system[C]// 2017 IEEE 22nd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks. Piscataway:IEEE Press, 2017: 1-6.
[9]	RAGI S , CHONG E K P . UAV path planning in a dynamic environment via partially observable markov decision process[J]. IEEE Transactions on Aerospace and Electronic Systems, 2013,49(4): 2397-2412.
[10]	ZHAO Y J , ZHENG Z , ZHANG X Y ,et al. Q learning algorithm based UAV path learning and obstacle avoidence approach[C]// 2017 36th Chinese Control Conference. Piscataway:IEEE Press, 2017: 3397-3402.
[11]	HUANG H , YANG Y , WANG H ,et al. Deep reinforcement learning for UAV navigption through massive MIMO technique[J]. IEEE Transactions on Vehicular Technology, 2020,69(1): 1117-1121.
[12]	WANG C , WANG J , SHEN Y ,et al. Autonomous navigption of UAVs in large-scale complex environments:a deep reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2019,PP(99): 1.
[13]	ZHANG B , LIU C H , TANG J ,et al. Learning-based energy-efficient data collection by unmanned vehicles in smart cities[J]. IEEE Transactions on Industrial Informatics, 2017,PP(99): 1.
[14]	LIU C H , CHEN Z , TANG J ,et al. Energy-efficient UAV control for effective and fair communication coverage:a deep reinforcement learning approach[J]. IEEE Journal on Selected Areas in Communications, 2018,36(9): 2059-2070.
[15]	NA L , SI-MING H , CHANG-QING G . UAV path planning based on adaptive weighted pigeon-inspired optimization algorithm[J]. Computer Simulation, 2018,35(1): 38-42,125.
[16]	ARANTES J D S , MARCIO D S A , TOLEDO C F M ,et al. Heuristic and genetic algorithm approaches for UAV path planning under critical situation[J]. International Journal of Artificial Intelligence Tools, 2017,26(1): 1760008.
[17]	刘丽珏, 罗舒宁, 高琰 ,等. 基于回溯蚁群-粒子群混合算法的多点路径规划[J]. 通信学报, 2019,40(2): 102-110.
	LIU L J , LUO S N , GAO Y ,et al. Multi-point path planning based on the algorithm of colony-particle swarm optimization[J]. Journal on Communications, 2019,40(2): 102-110.
[18]	VANSTEENWEGEN P , SOUFFRIAU W , OUDHEUSDEN D V . The orienteering problem:a survey[J]. European Journal of Operational Research, 2011,209(1): 1-10.
[19]	GUNAWAN A , LAU H C , VANSTEENWEGEN P . Orienteering problem:a survey of recent variants,solution approaches and applications[J]. European Journal of Operational Research, 2016,255(2): 315-332.
[20]	ABBASPOUR R A , SAMADZADEGAN F . Time-dependent personal tour planning and scheduling in metropolises[J]. Expert Systems with Applications, 2011,38(10): 12439-12452.
[21]	LI J , WU Q M , LI X Q ,et al. Study on the time-dependent orienteering problem[C]// International Conference on E-product E-service ＆ E-entertainment. Piscataway:IEEE Press, 2010: 1-4.
[22]	BRAHIM A , HASSAN E F . Iterated local search algorithm for solving the orienteering problem with soft time windows[J]. SpringerPlus, 2016,5(1): 1-36.
[23]	VINYALS O , FORTUNATO M , JAITLY N . Pointer networks[C]// International Conference on Neural Information Processing Systems. Massachusetts:MIT Press, 2015: 2692-2700.
[24]	BELLO I , PHAM H , LE Q V ,et al. Neural combinatorial optimization with reinforcement learning[J]. arXiv Preprint,arXiv:1611.09940v1, 2016.
[25]	HU H , ZHANG X , YAN X ,et al. Solving a new 3D bin packing problem with deep reinforcement learning method[J]. arXiv Preprint,arXiv:1708.05930, 2017.
[26]	王天荆, 李秀琴, 白光伟 ,等. 无线传感器网络中基于自适应网格的多目标定位算法[J]. 通信学报, 2019,40(7): 197-207.
	WANG T J , LI X Q , BAI G W ,et al. Multi-target localization algorithm based on adaptive grid in wireless sensor network[J]. Journal on Communications, 2019,40(7): 197-207.
[27]	WU S . Illegal radio station localization with UAV-based Q-learning[J]. China Communications, 2018,15(12): 122-131.
[28]	FARAJZADEH A , ERCETIN O . UAV data collection over NOMA backscatter networks:UAV altitude and trajectory optimization[J]. arXiv Preprint,arXiv:1902.03061, 2019.
[29]	JAWHAR I , MOHAMED N , AL-JAROODI J . UAV-based data communication in wireless sensor networks:models and strategies[C]// International Conference on Unmanned Aircraft Systems. Piscataway:IEEE Press, 2015: 687-694.
[30]	王继红, 石文孝 . 认知无线传感器网络分簇路由协议综述[J]. 通信学报, 2018,39(11): 156-169.
	WANG J H , SHI W X . Survey on cluster-based routing protocols for cognitive radio sensor networks[J]. Journal on Communications, 2018,39(11): 156-169.
[31]	KOOL W , VAN H H , WELLING M . Attention,learn to solve routing problems[J]. arXiv Preprint,arXiv:1803.08475v3, 2018.
[32]	3GPP. Further advancements for E-UTRA physical layer aspects[S]. TR 36.814 (V9.0.0), 2010.
[33]	PAN Y , DA X Y , HU H ,et al. Energy-efficiency optimization of UAV-based cognitive radio system[J]. IEEE Access, 2019,7: 155381-155391.
[34]	YAO Y , ZHU Z , HUANG S ,et al. Energy efficiency characterization in heterogeneous IoT system with UAV swarms based on wireless power transfer[J]. IEEE Access, 2020,8: 967-979.
[35]	ZHU Z , WANG N , HAO W ,et al. Robust beamforming designs in secure MIMO SWIPT IoT networks with a non-linear channel model[J]. IEEE Internet of Things Journal, 2020,PP(99): 1.
[36]	VINCENT F O , RAPHAEL F , DAMIEN E . Playing atari with deep reinforcement learning[J]. arXiv Preprint,arXiv:1312.5602, 2013.
[37]	SUTSKEVER I , VINYALS O , LE Q V . Sequence to Sequence Learning with Neural Networks[J]. Advances in Neural Information Processing Systems.Massachusetts:MIT Press, 2014: 3104-3112.
[38]	VINYALS O , KAISER L , KOO T ,et al. Grammar as a foreign language[J]. arXiv Preprint,arXiv:1412.7449, 2014.
[39]	HOCHREITER S , SCHMIDHUBER J . Long short-term memory[J]. Neural Computation, 1997,9(8): 1735-1780.
[40]	曾广贤 . 基于神经网络学习方法的单机调度问题研究[D]. 厦门:厦门大学, 2018.
	ZENG G X . Research on single machine scheduling problem based on neural network learning method[D]. Xiamen:Xiamen University, 2018.

参数	含义	值
H_s	PN的隐藏层数目	256
E_s	PN的embedding层数目	256
B	批次	256
T	迭代次数	100
γ	学习率	0.02
α	衰减系数	0.9
δ	簇总数份额	0.5

参数	含义	值
H_s	PN的隐藏层数目	128
E_s	PN的embedding层数目	128
B	批次	128
T	迭代次数	100
γ	学习率	0.02
α	衰减系数	0.9
δ	簇总数份额	0.5

仿真参数	取值
无人机初始位置	（20,20）
目标节点位置	（183,265）
步长/m	8
探索速率ε	0.1
传感器发射功率P _T /dBm	13
迭代次数T	100
更新周期t	30
记忆重放批次	400
折扣因子γ	0.9

任务	PN		AS		GP
任务	距离/km	奖励	距离/km	奖励	距离/km	奖励
D₅₀	11.71	12.89	6.45	17.78	5.71	17.82
D₁₀₀	24.56	19.32	10.87	33.68	7.08	35.07

UAV起点→目标节点坐标	Q学习最优步长/m	DQN最优步长/m
(20,20)→(183,265)	56	39
(10,10)→(388,378)	179	64
(0,0)→(68,78)	12	12

物联网数据收集中无人机路径智能规划

UAV path intelligent planning in IoT data collection

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 40

相关文章 15

Metrics

推荐阅读 0

[1]	周大成, 陈鸿昶, 何威振, 程国振, 扈红超. 基于深度强化学习的微服务多维动态防御策略研究[J]. 通信学报, 2023, 44(4): 50-63.
[2]	余雪勇, 邱礼翔, 宋家宁, 朱洪波. 无人机辅助边缘计算中安全通信与能效优化策略[J]. 通信学报, 2023, 44(3): 45-54.
[3]	夏景明, 刘玉风, 谈玲. 基于蜂窝网络的多无人机能量消耗最优化算法研究[J]. 通信学报, 2023, 44(2): 185-197.
[4]	王莉, 魏青, 徐连明, 沈渊, 张平, 费爱国. 面向通信-导航-感知一体化的应急无人机网络低能耗部署研究[J]. 通信学报, 2022, 43(7): 1-20.
[5]	郎磊, 王荆宁, 王一, 赵子涛. 无人机辅助通信中基于用户轨迹的无线资源和航迹优化[J]. 通信学报, 2022, 43(3): 225-232.
[6]	黄旭民, 张旸, 余荣, 蒋丽, 田辉, 吴远. 基于Stackelberg博弈的无人机辅助无线供能物联网能量优化[J]. 通信学报, 2022, 43(12): 146-156.
[7]	陈赓, 齐书虎, 沈斐, 曾庆田. 面向B5G多业务场景基于D3QN的双时间尺度网络切片算法[J]. 通信学报, 2022, 43(11): 213-224.
[8]	肖振宇, 刘珂, 朱立鹏. 无人机机间毫米波阵列通信技术[J]. 通信学报, 2022, 43(10): 196-209.
[9]	李斌, 刘文帅, 谢万城, 费泽松. 智能反射面赋能无人机边缘网络计算卸载方案[J]. 通信学报, 2022, 43(10): 223-233.
[10]	陈宇, 贾连兴. 面向无人机集群的双层分组拜占庭容错算法[J]. 通信学报, 2022, 43(1): 96-103.
[11]	杨君一, 李博, 张钦宇. 基于物理层网络编码的无人机中继网络资源优化[J]. 通信学报, 2021, 42(9): 12-20.
[12]	李赞, 胡俊凡, 李兵, 石嘉, 司江勃. 基于正交时频空技术的低轨卫星通信的安全分析[J]. 通信学报, 2021, 42(8): 25-32.
[13]	舒坚, 王启宁, 刘琳岚. 基于深度图嵌入的无人机自组网链路预测[J]. 通信学报, 2021, 42(7): 137-149.
[14]	徐思雅, 邢逸斐, 郭少勇, 杨超, 邱雪松, 孟洛明. 基于深度强化学习的能源互联网智能巡检任务分配机制[J]. 通信学报, 2021, 42(5): 191-204.
[15]	贺蕾, 马建峰, 魏大卫. 面向无人机网络的属性代理签名方案[J]. 通信学报, 2021, 42(11): 87-96.