物联网数据收集中无人机路径智能规划

doi:10.11959/j.issn.1000-436x.2021036

Abstract

Abstract:

To solve the problem of path planning of UAV data collection, it was generally be divided into global path planning and local path planning.For global path planning, it was modeled as an orientation problem, which was a combination of two classical optimization problems, the knapsack problem and the traveling salesman problem.The pointer network of deep learning was used to solve the model to obtain the service node set and service order under the energy constraint of the UAV.In terms of the local path planning, the reference signal strength (RSS) of the sensor node received by UAV was employed to learn the local flight path of UAV by deep Q network, which enabled the UAV to approach and serve the nodes.Simulation results show that the proposed scheme can effectively improve the revenue of UAV data collection under the energy constraint of UAV.

Key words: unmanned aerial vehicle, data collection, path planning, pointer network, deep Q network

CLC Number:

TN92

Shu FU, Xiangyue YANG, Haijun ZHANG, Chen CHEN, Peng YU, Xin JIAN, Min LIU. UAV path intelligent planning in IoT data collection[J]. Journal on Communications, 2021, 42(2): 124-133.

Figures/Tables 15

References 40

[1]	LI C , ZHANG H , HAO B ,et al. A survey on routing protocols for large-scale wireless sensor networks[J]. Sensors, 2011,11(12): 3498-3526.
[2]	ZENG Y , ZHANG R , LIM T J . Wireless communications with unmanned aerial vehicles:opportunities and challenges[J]. IEEE Communications Magazine, 2016,54(5): 36-42.
[3]	WANG C , MA F , YAN J ,et al. Efficient aerial data collection with UAV in large-scale wireless sensor networks[J]. International Journal of Distributed Sensor Networks, 2015,2015: 1-19.
[4]	UTKARSHA S.P , RAJIV K G . Clustering and compressive data gpthering in wireless sensor network[J]. Wireless Personal Communications, 2019,109(2): 1-21.
[5]	JIE G , TSUNG-HUI C , CHAO S ,et al. Flight time minimization of UAV for data collection over wireless sensor networks[J]. IEEE Journal on Selected Areas in Communications, 2018,PP(99): 1.
[6]	ZAIN A A , SUHAIB M , MUHAMMAD A . UAV based data gpthering in wireless sensor networks[J]. Wireless Personal Communications, 2019,106(4): 1-11.
[7]	DUAN H , LI P . UAV path planning[M]. Berlin: Springer, 2014.
[8]	ARVANITAKI A , PAPPAS N . Modeling of a UAV-based data collection system[C]// 2017 IEEE 22nd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks. Piscataway:IEEE Press, 2017: 1-6.
[9]	RAGI S , CHONG E K P . UAV path planning in a dynamic environment via partially observable markov decision process[J]. IEEE Transactions on Aerospace and Electronic Systems, 2013,49(4): 2397-2412.
[10]	ZHAO Y J , ZHENG Z , ZHANG X Y ,et al. Q learning algorithm based UAV path learning and obstacle avoidence approach[C]// 2017 36th Chinese Control Conference. Piscataway:IEEE Press, 2017: 3397-3402.
[11]	HUANG H , YANG Y , WANG H ,et al. Deep reinforcement learning for UAV navigption through massive MIMO technique[J]. IEEE Transactions on Vehicular Technology, 2020,69(1): 1117-1121.
[12]	WANG C , WANG J , SHEN Y ,et al. Autonomous navigption of UAVs in large-scale complex environments:a deep reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2019,PP(99): 1.
[13]	ZHANG B , LIU C H , TANG J ,et al. Learning-based energy-efficient data collection by unmanned vehicles in smart cities[J]. IEEE Transactions on Industrial Informatics, 2017,PP(99): 1.
[14]	LIU C H , CHEN Z , TANG J ,et al. Energy-efficient UAV control for effective and fair communication coverage:a deep reinforcement learning approach[J]. IEEE Journal on Selected Areas in Communications, 2018,36(9): 2059-2070.
[15]	NA L , SI-MING H , CHANG-QING G . UAV path planning based on adaptive weighted pigeon-inspired optimization algorithm[J]. Computer Simulation, 2018,35(1): 38-42,125.
[16]	ARANTES J D S , MARCIO D S A , TOLEDO C F M ,et al. Heuristic and genetic algorithm approaches for UAV path planning under critical situation[J]. International Journal of Artificial Intelligence Tools, 2017,26(1): 1760008.
[17]	刘丽珏, 罗舒宁, 高琰 ,等. 基于回溯蚁群-粒子群混合算法的多点路径规划[J]. 通信学报, 2019,40(2): 102-110.
	LIU L J , LUO S N , GAO Y ,et al. Multi-point path planning based on the algorithm of colony-particle swarm optimization[J]. Journal on Communications, 2019,40(2): 102-110.
[18]	VANSTEENWEGEN P , SOUFFRIAU W , OUDHEUSDEN D V . The orienteering problem:a survey[J]. European Journal of Operational Research, 2011,209(1): 1-10.
[19]	GUNAWAN A , LAU H C , VANSTEENWEGEN P . Orienteering problem:a survey of recent variants,solution approaches and applications[J]. European Journal of Operational Research, 2016,255(2): 315-332.
[20]	ABBASPOUR R A , SAMADZADEGAN F . Time-dependent personal tour planning and scheduling in metropolises[J]. Expert Systems with Applications, 2011,38(10): 12439-12452.
[21]	LI J , WU Q M , LI X Q ,et al. Study on the time-dependent orienteering problem[C]// International Conference on E-product E-service ＆ E-entertainment. Piscataway:IEEE Press, 2010: 1-4.
[22]	BRAHIM A , HASSAN E F . Iterated local search algorithm for solving the orienteering problem with soft time windows[J]. SpringerPlus, 2016,5(1): 1-36.
[23]	VINYALS O , FORTUNATO M , JAITLY N . Pointer networks[C]// International Conference on Neural Information Processing Systems. Massachusetts:MIT Press, 2015: 2692-2700.
[24]	BELLO I , PHAM H , LE Q V ,et al. Neural combinatorial optimization with reinforcement learning[J]. arXiv Preprint,arXiv:1611.09940v1, 2016.
[25]	HU H , ZHANG X , YAN X ,et al. Solving a new 3D bin packing problem with deep reinforcement learning method[J]. arXiv Preprint,arXiv:1708.05930, 2017.
[26]	王天荆, 李秀琴, 白光伟 ,等. 无线传感器网络中基于自适应网格的多目标定位算法[J]. 通信学报, 2019,40(7): 197-207.
	WANG T J , LI X Q , BAI G W ,et al. Multi-target localization algorithm based on adaptive grid in wireless sensor network[J]. Journal on Communications, 2019,40(7): 197-207.
[27]	WU S . Illegal radio station localization with UAV-based Q-learning[J]. China Communications, 2018,15(12): 122-131.
[28]	FARAJZADEH A , ERCETIN O . UAV data collection over NOMA backscatter networks:UAV altitude and trajectory optimization[J]. arXiv Preprint,arXiv:1902.03061, 2019.
[29]	JAWHAR I , MOHAMED N , AL-JAROODI J . UAV-based data communication in wireless sensor networks:models and strategies[C]// International Conference on Unmanned Aircraft Systems. Piscataway:IEEE Press, 2015: 687-694.
[30]	王继红, 石文孝 . 认知无线传感器网络分簇路由协议综述[J]. 通信学报, 2018,39(11): 156-169.
	WANG J H , SHI W X . Survey on cluster-based routing protocols for cognitive radio sensor networks[J]. Journal on Communications, 2018,39(11): 156-169.
[31]	KOOL W , VAN H H , WELLING M . Attention,learn to solve routing problems[J]. arXiv Preprint,arXiv:1803.08475v3, 2018.
[32]	3GPP. Further advancements for E-UTRA physical layer aspects[S]. TR 36.814 (V9.0.0), 2010.
[33]	PAN Y , DA X Y , HU H ,et al. Energy-efficiency optimization of UAV-based cognitive radio system[J]. IEEE Access, 2019,7: 155381-155391.
[34]	YAO Y , ZHU Z , HUANG S ,et al. Energy efficiency characterization in heterogeneous IoT system with UAV swarms based on wireless power transfer[J]. IEEE Access, 2020,8: 967-979.
[35]	ZHU Z , WANG N , HAO W ,et al. Robust beamforming designs in secure MIMO SWIPT IoT networks with a non-linear channel model[J]. IEEE Internet of Things Journal, 2020,PP(99): 1.
[36]	VINCENT F O , RAPHAEL F , DAMIEN E . Playing atari with deep reinforcement learning[J]. arXiv Preprint,arXiv:1312.5602, 2013.
[37]	SUTSKEVER I , VINYALS O , LE Q V . Sequence to Sequence Learning with Neural Networks[J]. Advances in Neural Information Processing Systems.Massachusetts:MIT Press, 2014: 3104-3112.
[38]	VINYALS O , KAISER L , KOO T ,et al. Grammar as a foreign language[J]. arXiv Preprint,arXiv:1412.7449, 2014.
[39]	HOCHREITER S , SCHMIDHUBER J . Long short-term memory[J]. Neural Computation, 1997,9(8): 1735-1780.
[40]	曾广贤 . 基于神经网络学习方法的单机调度问题研究[D]. 厦门:厦门大学, 2018.
	ZENG G X . Research on single machine scheduling problem based on neural network learning method[D]. Xiamen:Xiamen University, 2018.

Metrics

Recommended 0

No Suggested Reading articles found!

参数	含义	值
H_s	PN的隐藏层数目	256
E_s	PN的embedding层数目	256
B	批次	256
T	迭代次数	100
γ	学习率	0.02
α	衰减系数	0.9
δ	簇总数份额	0.5

参数	含义	值
H_s	PN的隐藏层数目	128
E_s	PN的embedding层数目	128
B	批次	128
T	迭代次数	100
γ	学习率	0.02
α	衰减系数	0.9
δ	簇总数份额	0.5

仿真参数	取值
无人机初始位置	（20,20）
目标节点位置	（183,265）
步长/m	8
探索速率ε	0.1
传感器发射功率P _T /dBm	13
迭代次数T	100
更新周期t	30
记忆重放批次	400
折扣因子γ	0.9

任务	PN		AS		GP
任务	距离/km	奖励	距离/km	奖励	距离/km	奖励
D₅₀	11.71	12.89	6.45	17.78	5.71	17.82
D₁₀₀	24.56	19.32	10.87	33.68	7.08	35.07

UAV起点→目标节点坐标	Q学习最优步长/m	DQN最优步长/m
(20,20)→(183,265)	56	39
(10,10)→(388,378)	179	64
(0,0)→(68,78)	12	12

UAV path intelligent planning in IoT data collection

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 40

Related Articles 15

Metrics

Recommended 0

[1]	Zhenyu XIAO, Ke LIU, Lipeng ZHU. Millimeter-wave array enabled UAV-to-UAV communication technology [J]. Journal on Communications, 2022, 43(10): 196-209.
[2]	Lei HE, Jianfeng MA, Dawei WEI. Attribute-based proxy signature scheme for unmanned aerial vehicle networks [J]. Journal on Communications, 2021, 42(11): 87-96.
[3]	Haitao ZHAO,Shishun GAO,Haijun WANG,Ting YONG,Jibo WEI. Evaluation method for autonomous communication and networking capability of UAV [J]. Journal on Communications, 2020, 41(8): 87-98.
[4]	Jiazhi REN,Hui TIAN,Shaoshuai FAN,Yuanzhuo LIN,Gaofeng NIE,Jilong LI. UAV deployment and caching scheme based on user preference prediction [J]. Journal on Communications, 2020, 41(6): 1-13.
[5]	Chaoqiong FAN,Chenglin ZHAO,Bin LI. Hierarchical game based spectrum access optimization for anti-jamming in UAV network [J]. Journal on Communications, 2020, 41(6): 26-33.
[6]	Bonan YIN,Mugen PENG,Chenxi LIU. Analysis of UAV wireless network coverage and handover performance [J]. Journal on Communications, 2020, 41(11): 22-29.
[7]	Fenghua LI,Zifu LI,Ling LI,Ming ZHANG,Kui GENG,Yunchuan GUO. Collection policy refining method for threat monitoring in complex network environment [J]. Journal on Communications, 2019, 40(4): 49-61.
[8]	Bo ZHENG,Hengyang ZHANG,Yong LI,Wei CHENG. Coverage in airborne backbone network [J]. Journal on Communications, 2018, 39(11): 36-43.
[9]	Zhenchun WEI,Renhao SUN,Zengwei LYU,Jianghong HAN,Lei SHI,Junyi XU. Path planning algorithm for WCE with joint energy replenishment and data collection based on multi-objective optimization [J]. Journal on Communications, 2018, 39(10): 22-33.
[10]	Wei-ping ZHANG,Ya-hong GUO,Meng WANG,Lin-yu NI,Jin-bao LI. Forest based data collection in MR-MC wireless sensor networks [J]. Journal on Communications, 2016, 37(3): 40-47.
[11]	Xian-ling LU,Ying-ying WANG. Data collection algorithm for mobile sink in delay-constrained network [J]. Journal on Communications, 2014, 35(10): 107-116.
[12]	. Data collection algorithm for mobile sink in delay-constrained network [J]. Journal on Communications, 2014, 35(10): 13-116.
[13]	Jun YUE,Wei-ming ZHANG,Wei-dong XIAO,Da-quan TANG,Jiu-yang TANG. Energy dissipation balanced and reliable clustering algorithm for wireless sensor networks [J]. Journal on Communications, 2012, 33(Z2): 90-96.
[14]	Jian GUO,Li-juan SUN,Wen-jun XU,Ru-chuan WANG,Fu XIAO. Mobile sink-based data collection scheme for wireless sensor networks [J]. Journal on Communications, 2012, 33(9): 176-184.
[15]	Huan-qing CUI,Ying-long WANG,Qiang GUO,Jia-liang LV. Multi-mobile-beacon assisted distributed node localization scheme [J]. Journal on Communications, 2012, 33(3): 103-111.