面向可伸缩视频编码传输的DDPG无人机服务增强机制

doi:10.11959/j.issn.1000-0801.2023158

电信科学 ›› 2023, Vol. 39 ›› Issue (8): 69-81.doi: 10.11959/j.issn.1000-0801.2023158

• 研究与开发 • 上一篇

面向可伸缩视频编码传输的DDPG无人机服务增强机制

闫俊杰¹, 朱亚新¹, 冯艳茹², 刘汉永³, 邓钧忆¹, 王欢¹

¹ 广西科技大学电子工程学院，广西柳州 545006
² 山西应用科技学院，山西太原 030062
³ 国家电网重庆市电力公司大足供电分公司，重庆 402360

修回日期:2023-08-09 出版日期:2023-08-01 发布日期:2023-08-01
作者简介:闫俊杰（1990- ），男，博士，广西科技大学电子工程学院讲师，主要研究方向为MEC、UAV通信、D2D 通信
朱亚新（1998- ），女，广西科技大学电子工程学院硕士生，主要研究方向为 MEC、UAV通信、D2D通信
冯艳如（1990- ），女，山西应用科技学院讲师，主要研究方向为信息传输及信息安全
刘汉永（1992- ），女，国家电网重庆市电力公司大足供电分公司工程师，主要研究方向为区块链、网络安全、密码学、自动化、电网营销
邓钧忆（1985- ），男，博士，广西科技大学电子工程学院助理研究员，主要研究方向为车联网、MEC
王欢（1987- ），男，博士，广西科技大学电子工程学院副教授，主要研究方向为智能网络与信息安全、高可用软件
基金资助:
国家自然科学基金资助项目(62106053);广西自然科学基金资助项目(2022GXNSFBA035642);广西科技计划项目(2021AC19167);广西中青年教师基础能力提升项目(2022KY0325);广西中青年教师基础能力提升项目(2021KY0364)

UAV service enhancement mechanism deep deterministic policy gradient for scalable video coding transmission

Junjie YAN¹, Yaxin ZHU¹, Yanru FENG², Hanyong LIU³, Junyi DENG¹, Huan WANG¹

¹ School of Electronic Engineering, Guangxi University of Science and Technology, Liuzhou 545006, China
² Shanxi College of Applied Science and Technology, Taiyuan 030062, China
³ Dazu Power Supply Company of State Grid Chongqing Electric Power Company, Chongqing 402360, China

Revised:2023-08-09 Online:2023-08-01 Published:2023-08-01
Supported by:
The National Natural Science Foundation of China(62106053);The Natural Science Foundation of Guangxi(2022GXNSFBA035642);Guangxi Science and Technology Planning Project(2021AC19167);The Basic Ability Enhancement Program for Young and Middle-aged Teachers of Guangxi(2022KY0325);The Basic Ability Enhancement Program for Young and Middle-aged Teachers of Guangxi(2021KY0364)

摘要/Abstract

摘要：

无人机（UAV）因其机载存储、通信和计算能力成为未来空天地一体化网络的重要组成部分。然而，现有无人机辅助的边缘网络研究总是更多地从网络的视角出发，缺少从用户视角出发带来的变革。为此，从用户角度出发，提出一种面向可伸缩视频传输的DDPG无人机服务增强机制。首先，结合无人机提出一种基于可伸缩视频编码（SVC）的弹性视频传输方法，提高用户的差异化体验。其次，以最大化用户接收增强层视频的数量为目标，提出一种基于深度确定性策略梯度（DDPG）算法的无人机路径规划设计，保证UAV对热点区域增强覆盖的有效性。仿真结果表明，所提出的算法与深度Q网络（DQN）算法、最短路径（SP）算法相比，在不同的用户分布下得到的增强层数量平均分别可以提高47.9%、76.4%，该研究达到了热点区域覆盖增强和用户差异化体验的效果。

关键词: 深度确定性策略梯度, 无人机, 可伸缩视频编码, 路径规划

Abstract:

Unmanned aerial vehicles (UAV) are an important component of future air-space-ground integrated network due to their onboard storage, communication, and computing capabilities.However, existing research on UAV-assisted edge networks often focuses more on the network perspective and lacks consideration of user perspectives and their requirements.Therefore, a UAV service enhancement mechanism based on deep deterministic policy gradient (DDPG) for scalable video coding (SVC) transmission was proposed from the user’s perspective.Firstly, an elastic video transmission method based on SVC was proposed in conjunction with UAV to improve differentiated user experiences.Secondly, a UAV trajectory planning design based on the DDPG algorithm was proposed to maximize the number of users receiving enhanced layer videos and ensure effective coverage enhancement by UAV in hotspot areas.Simulation results show that compared with both the deep Q network (DQN) algorithm and shortest path (SP) algorithm under different user distributions, the proposed algorithm can increase the average number of enhanced layers received by 47.9% and 76.4%, respectively.This study successfully achieves improved coverage in hotspot areas while also providing differentiated user experiences through its proposed methods.

Key words: DDPG, UAV, SVC, trajectory planning

中图分类号:

TN929.5

闫俊杰, 朱亚新, 冯艳茹, 刘汉永, 邓钧忆, 王欢. 面向可伸缩视频编码传输的DDPG无人机服务增强机制[J]. 电信科学, 2023, 39(8): 69-81.

Junjie YAN, Yaxin ZHU, Yanru FENG, Hanyong LIU, Junyi DENG, Huan WANG. UAV service enhancement mechanism deep deterministic policy gradient for scalable video coding transmission[J]. Telecommunications Science, 2023, 39(8): 69-81.

图/表 10

图1

图2

图3

表1

模型仿真参数"

参数	值
用户数Z	100
UAV高度H	100 m
用户范围和UAV轨迹范围 ${x_{u}^{max}, y_{u}^{max}}$	[400 m, 400 m]
1 m参考距离的信道增益 $ρ_{u}$ 、 $ρ_{f}$	-30 dB
路径损耗指数α	2
带宽B	10 MHz
U AV的传输功率P_{u m}	1W
UAV的速度v_max	20 m/s
奖励函数中的缩放常数κ	1×10^-2
奖励函数中的缩放常数λ	1×10^-3
SBS与基本层用户的速率阈值R_B	0.442 Mbit/s
UAV与增强层1的用户速率阈值R_E1	0.827 Mbit/s
UAV与增强层2的用户速率阈值R_E2	1.114 Mbit/s

表1

图4

图5

图6

图7

图8

图9

参考文献 20

[1]	WU Q Q , ZENG Y , ZHANG R . Joint trajectory and communication design for multi-UAV enabled wireless networks[J]. IEEE Transactions on Wireless Communications, 2018,17(3): 2109-2121.
[2]	FU Y J , MEI H B , WANG K Z ,et al. Joint optimization of 3D trajectory and scheduling for solar-powered UAV systems[J]. IEEE Transactions on Vehicular Technology, 2021,70(4): 3972-3977.
[3]	陈新颖, 盛敏, 李博 ,等. 面向 6G 的无人机通信综述[J]. 电子与信息学报, 2022,44(3): 781-789.
	CHEN X Y , SHENG M , LI B ,et al. Survey on unmanned aerial vehicle communications for 6G[J]. Journal of Electronics ＆ Information Technology, 2022,44(3): 781-789.
[4]	ZHOU H , HU F H , JURAS M ,et al. Real-time video streaming and control of cellular-connected UAV system:prototype and performance evaluation[J]. IEEE Wireless Communications Letters, 2021,10(8): 1657-1661.
[5]	ZHAN C , HUANG R J . Energy efficient adaptive video streaming with rotary-wing UAV[J]. IEEE Transactions on Vehicular Technology, 2020,69(7): 8040-8044.
[6]	ZHAO N , CHENG F , YU F R ,et al. Caching UAV assisted secure transmission in hyper-dense networks based on interference alignment[J]. IEEE Transactions on Communications, 2018,66(5): 2281-2294.
[7]	LIU Z C , JIANG Y . Cross-layer design for UAV-based streaming media transmission[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022,32(7): 4710-4723.
[8]	ZHANG Y Q , XU C , HEMADEH I A ,et al. Near-instantaneously adaptive multi-set space-time shift keying for UAV-aided video surveillance[J]. IEEE Transactions on Vehicular Technology, 2020,69(11): 12843-12856.
[9]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.
[10]	XIAO L , DING Y Z , HUANG J H ,et al. UAV anti-jamming video transmissions with QoE guarantee:a reinforcement learning-based approach[J]. IEEE Transactions on Communications, 2021,69(9): 5933-5947.
[11]	LIU Y , YAN J J , ZHAO X H . Deep reinforcement learning based latency minimization for mobile edge computing with virtualization in maritime UAV communication network[J]. IEEE Transactions on Vehicular Technology, 2022,71(4): 4225-4236.
[12]	DING R J , GAO F F , SHEN X S . 3D UAV trajectory design and frequency band allocation for energy-efficient and fair communication:a deep reinforcement learning approach[J]. IEEE Transactions on Wireless Communications, 2020,19(12): 7796-7809.
[13]	CUI Y L , DENG D H , WANG C W ,et al. Joint trajectory and power optimization for energy efficient UAV communication using deep reinforcement learning[C]// Proceedings of IEEE INFOCOM 2021 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). Piscataway:IEEE Press, 2021: 1-6.
[14]	SAMIR M , ASSI C , SHARAFEDDINE S ,et al. Age of information aware trajectory planning of UAVs in intelligent transportation systems:a deep learning approach[J]. IEEE Transactions on Vehicular Technology, 2020,69(11): 12382-12395.
[15]	KANG H Y , CHANG X L , MI?I? J , ,et al. Improving dual-UAV aided ground-UAV Bi-directional communication security:joint UAV trajectory and transmit power optimization[J]. IEEE Transactions on Vehicular Technology, 2022,71(10): 10570-10583.
[16]	WU D P , LIU Q R , WANG H G ,et al. Cache less for more:exploiting cooperative video caching and delivery in D2D communications[J]. IEEE Transactions on Multimedia, 2019,21(7): 1788-1798.
[17]	SONG D , ZHAI X B , LIU X ,et al. Jointly optimal fair data collection and trajectory design algorithms in UAV-aided cellular networks[C]// Proceedings of 2021 IEEE Wireless Communications and Networking Conference (WCNC). Piscataway:IEEE Press, 2021: 1-6.
[18]	CONDOLUCI M , ARANITI G , MOLINARO A ,et al. Multicast resource allocation enhanced by channel state feedbacks for multiple scalable video coding streams in LTE networks[J]. IEEE Transactions on Vehicular Technology, 2016,65(5): 2907-2921.
[19]	ZENG Y , XU J , ZHANG R . Energy minimization for wireless communication with rotary-wing UAV[J]. IEEE Transactions on Wireless Communications, 2019,18(4): 2329-2345.
[20]	TANG F X , HOFNER H , KATO N ,et al. A deep reinforcement learning-based dynamic traffic offloading in space-air-ground integrated networks (SAGIN)[J]. IEEE Journal on Selected Areas in Communications, 2022,40(1): 276-289.

面向可伸缩视频编码传输的DDPG无人机服务增强机制

UAV service enhancement mechanism deep deterministic policy gradient for scalable video coding transmission

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 20

相关文章 15

Metrics

推荐阅读 0

[1]	白恒志, 王海超, 李国鑫, 龚玉萍. 无人机隐蔽通信网络研究综述[J]. 电信科学, 2023, 39(8): 1-16.
[2]	何耀宇, 张超. 面向无人机应用的低轨卫星通信技术适航分析[J]. 电信科学, 2023, 39(6): 96-104.
[3]	刘晨熙, 马睿, 彭木根. 无人机通信感知一体化：架构、技术与展望[J]. 电信科学, 2023, 39(2): 1-9.
[4]	韩慧, 郝玉龙, 杨铖, 王健. 基于抛物方程的无人机通信场景传播特性分析[J]. 电信科学, 2022, 38(8): 45-53.
[5]	邓丹昊, 王朝炜, 江帆, 王卫东. 无人机辅助无蜂窝大规模MIMO中的空地协同调度[J]. 电信科学, 2022, 38(8): 37-44.
[6]	丁雨, 李晨凯, 韩会梅, 卢为党, 任元红, 高原, 曹江. 基于5G无人机通信的多智能体异构网络选择方法[J]. 电信科学, 2022, 38(8): 28-36.
[7]	曾子鸣, 董超, 朱小军, 贾子晔. 面向群体快速融合的仿生无人机集群架构[J]. 电信科学, 2022, 38(8): 17-27.
[8]	张天魁, 徐瑜, 刘元玮, 杨鼎成, 任元红. 无人机辅助MEC系统：架构、关键技术与未来挑战[J]. 电信科学, 2022, 38(8): 3-16.
[9]	徐溯, 张际, 刁杨华, 刘元莹, 张懿. 基于混合光电传输的无人机中继通信系统性能分析[J]. 电信科学, 2022, 38(4): 113-120.
[10]	李远宁,宁柏锋,董召杰. 电网机巡图像分析框架与深度学习方法[J]. 电信科学, 2020, 36(8): 167-174.
[11]	淡振雷,吴雪雯,欧阳键,汪萧萧,顾晨伟,林敏. 基于门限判断DF协议的多无人机中继传输中断性能分析[J]. 电信科学, 2020, 36(8): 103-111.
[12]	马小婷,赵军辉,孙笑科,贡毅. 基于MEC的车联网协作组网关键技术[J]. 电信科学, 2020, 36(6): 28-37.
[13]	张勍,冯毅,马丹,吕琦. 5G低空网络解决方案和运营应用[J]. 电信科学, 2020, 36(1): 28-33.
[14]	周峰,王兵. 基于人工智能的基站巡检应用实践[J]. 电信科学, 2018, 34(12): 102-109.
[15]	贺子健,艾元,闫实,彭木根. 无人机通信网络的容量与覆盖性能[J]. 电信科学, 2017, 33(10): 65-70.