基于深度强化学习的可见光定位通信一体化功率分配研究

doi:10.11959/j.issn.1000-436x.2022163

通信学报 ›› 2022, Vol. 43 ›› Issue (8): 121-130.doi: 10.11959/j.issn.1000-436x.2022163

基于深度强化学习的可见光定位通信一体化功率分配研究

马帅, 李兵, 盛海鸿, 谷荣妍, 周辉, 王洪梅, 王悦, 李世银

中国矿业大学信息与控制工程学院，江苏徐州 221000

修回日期:2022-08-08 出版日期:2022-08-25 发布日期:2022-08-01
作者简介:马帅（1986- ），男，山东日照人，博士，中国矿业大学副教授，主要研究方向为语义通信和可见光通信定位一体化等
李兵（1997- ），男，河南商丘人，中国矿业大学硕士生，主要研究方向为可见光通信和可见光定位
盛海鸿（1998- ），女，江苏泰州人，中国矿业大学硕士生，主要研究方向为无线通信和可见光通信
谷荣妍（1999- ），女，安徽六安人，中国矿业大学硕士生，主要研究方向为可见光通信和可见光定位
周辉（1997- ），男，湖北荆州人，中国矿业大学硕士生，主要研究方向为可见光通信和定位通信一体化
王洪梅（1983- ），女，山东诸城人，博士，中国矿业大学副教授，主要研究方向为无线通信
王悦（1994- ），女，吉林四平人，博士，中国矿业大学副教授，主要研究方向为光通信、微波光子学、集成光子器件等
李世银（1971- ），男，四川犍为人，博士，中国矿业大学教授、博士生导师，主要研究方向为煤矿信息化和移动目标定位
基金资助:
中央高校基本科研业务费专项资金资助项目(2022QN1052);江苏省自然科学基金资助项目(BK20221115);中国矿业大学未来科学家计划基金资助项目(2022WLJCRCZL108)

Research on power allocation of integrated VLPC based on deep reinforcement learning

Shuai MA, Bing LI, Haihong SHENG, Rongyan GU, Hui ZHOU, Hongmei WANG, Yue WANG, Shiyin LI

School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221000, China

Revised:2022-08-08 Online:2022-08-25 Published:2022-08-01
Supported by:
The Fundamental Research Funds for the Central Universities(2022QN1052);The Natural Science Foundation of Jiangsu Province(BK20221115);Funded by the Graduate Innovation Program of China University of Mining and Technology(2022WLJCRCZL108)

摘要/Abstract

摘要：

为了实现定位通信一体化功率分配，提出了一种基于深度强化学习的可见光定位通信（VLPC）一体化系统的功率分配方案。首先，提出了定位通信一体化帧结构设计；其次，利用定位信息实现了信道状态信息的估计，并推导了定位误差的克拉美罗下界（CRLB）；再次，阐明了定位精度和通信速率的内在耦合关系；最后，提出了基于深度确定性策略梯度的VLPC动态功率分配方案。仿真结果表明，所提方案可同时实现高精度定位和高速通信。

关键词: 可见光定位通信一体化, 克拉美罗下界, 功率分配, 强化学习

Abstract:

A power allocation scheme for integrated visible light position and communication (VLPC) system based on deep reinforcement learning was proposed to achieve power allocation for communication positioning integration.First, the frame structure design of integrated VLPC was proposed.Then the channel state information could be estimated by using the positioning information, and the CRLB of the positioning error was derived.Furthermore, the internal coupling relationship between positioning accuracy and communication rate was clarified.On this basis, a dynamic power allocation scheme based on deep deterministic policy gradient was proposed.Simulation results show that the proposed scheme can simultaneously achieve high-precision positioning and high-speed communication.

Key words: integrated VLPC, Cramér-Rao lower bound, power allocation, reinforcement learning

中图分类号:

TN92

马帅, 李兵, 盛海鸿, 谷荣妍, 周辉, 王洪梅, 王悦, 李世银. 基于深度强化学习的可见光定位通信一体化功率分配研究[J]. 通信学报, 2022, 43(8): 121-130.

Shuai MA, Bing LI, Haihong SHENG, Rongyan GU, Hui ZHOU, Hongmei WANG, Yue WANG, Shiyin LI. Research on power allocation of integrated VLPC based on deep reinforcement learning[J]. Journal on Communications, 2022, 43(8): 121-130.

图/表 9

图1

图2

图3

表1

VLPC系统参数"

参数	参数值
LED半功率角 $ψ_{\frac{1}{2}}$	60^D
接收机视场角?_FOV	90^D
折射率n_r	1.5
接收机感光面积A_r/cm²	1.0
聚光器增益Γ(?)/(A·W^-1)	1.0
滤光片增益F_s (A·W^-1)	1.0
噪声功率谱密度 $σ_{p}^{2}, σ_{c}^{2}$ /dBm	-98.82
信号幅度A/V	1.0
信号方差ε	1.0
直流偏置b/A	0.447 2
平均电功率P_e/W	1.0
最大光功率 $P_{o}^{\max}$ /W	2.236
总功率门限 $P_{total}^{}$ /W	3

表1

表2

图4

图5

图6

图7

参考文献 24

[1]	LIU G Y , HUANG Y H , LI N ,et al. Vision,requirements and network architecture of 6G mobile network beyond 2030[J]. China Communications, 2020,17(9): 92-104.
[2]	迟楠, 陈慧 . 高速可见光通信的前沿研究进展[J]. 光电工程, 2020,47(3): 190687.
	CHI N , CHEN H . Progress and prospect of high-speed visible light communication[J]. Opto-Electronic Engineering, 2020,47(3): 190687.
[3]	DAI L L , WANG B C , YUAN Y F ,et al. Non-orthogonal multiple access for 5G:solutions,challenges,opportunities,and future research trends[J]. IEEE Communications Magazine, 2015,53(9): 74-81.
[4]	JIANG W , HAN B , HABIBI M A ,et al. The road towards 6G:a comprehensive survey[J]. IEEE Open Journal of the Communications Society, 2021,2: 334-366.
[5]	MA S , LI H , HE Y ,et al. Capacity bounds and interference management for interference channel in visible light communication networks[J]. IEEE Transactions on Wireless Communications, 2019,18(1): 182-193.
[6]	JIANG R , WANG Q , HAAS H ,et al. Joint user association and power allocation for cell-free visible light communication networks[J]. IEEE Journal on Selected Areas in Communications, 2018,36(1): 136-148.
[7]	SHI J Y , ZHU X , WANG F M ,et al. Net data rate of 14.6 Gbit/s underwater VLC utilizing silicon substrate common-anode five primary colors LED[C]// Proceedings of Optical Fiber Communication Conference. Washington:OSA, 2019: 1-3.
[8]	YANG S H , JUNG E M , HAN S K . Indoor location estimation based on LED visible light communication using multiple optical receivers[J]. IEEE Communications Letters, 2013,17(9): 1834-1837.
[9]	YANG S H , KIM H S , SON Y H ,et al. Three-dimensional visible light indoor localization using AOA and RSS with multiple optical receivers[J]. Journal of Lightwave Technology, 2014,32(14): 2480-2485.
[10]	WU Y C , HSU K L , LIU Y ,et al. Using linear interpolation to reduce the training samples for regression based visible light positioning system[J]. IEEE Photonics Journal, 2020,12(2): 1-5.
[11]	YANG H L , ZHONG W D , CHEN C ,et al. QoS-driven optimized design-based integrated visible light communication and positioning for indoor IoT networks[J]. IEEE Internet of Things Journal, 2020,7(1): 269-283.
[12]	LIN B J , TANG X , GHASSEMLOOY Z ,et al. Experimental demonstration of an indoor VLC positioning system based on OFDMA[J]. IEEE Photonics Journal, 2017,9(2): 1-9.
[13]	YANG H L , CHEN C , ZHONG W D ,et al. Demonstration of a quasi-gapless integrated visible light communication and positioning system[J]. IEEE Photonics Technology Letters, 2018,30(23): 2001-2004.
[14]	YANG H L , DU P F , ZHONG W D ,et al. Reinforcement learning-based intelligent resource allocation for integrated VLCP systems[J]. IEEE Wireless Communications Letters, 2019,8(4): 1204-1207.
[15]	WANG X M , ZHANG Y H , SHEN R J ,et al. DRL-based energy-efficient resource allocation frameworks for uplink NOMA systems[J]. IEEE Internet of Things Journal, 2020,7(8): 7279-7294.
[16]	CHU M , LI H , LIAO X W ,et al. Reinforcement learning-based multiaccess control and battery prediction with energy harvesting in IoT systems[J]. IEEE Internet of Things Journal, 2019,6(2): 2009-2020.
[17]	KAHN J M , BARRY J R . Wireless infrared communications[J]. Proceedings of the IEEE, 1997,85(2): 265-298.
[18]	STEVENS N , STEENDAM H . Magnitude of the distance estimation bias in received signal strength visible light positioning[J]. IEEE Communications Letters, 2018,22(11): 2250-2253.
[19]	KESKIN M F , SEZER A D , GEZICI S . Optimal and robust power allocation for visible light positioning systems under illumination constraints[J]. IEEE Transactions on Communications, 2019,67(1): 527-542.
[20]	MA S , ZHANG F , LI H ,et al. Simultaneous lightwave information and power transfer in visible light communication systems[J]. IEEE Transactions on Wireless Communications, 2019,18(12): 5818-5830.
[21]	周志华 . 机器学习[M]. 北京: 清华大学出版社, 2016.
	ZHOU Z H . Machine learning[M]. Beijing: Tsinghua University Press, 2016.
[22]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.
[23]	XU Y H , YANG C C , HUA M ,et al. Deep deterministic policy gradient (DDPG)-based resource allocation scheme for NOMA vehicular communications[J]. IEEE Access, 2020,8: 18797-18807.
[24]	CALABRESE F D , WANG L , GHADIMI E ,et al. Learning radio resource management in RANs:framework,opportunities,and challenges[J]. IEEE Communications Magazine, 2018,56(9): 138-145.

参数	参数值
回合数E	1 000
每回合总时隙T	20
经验池D大小	3 000
采样批次Z	32
优化器	Adam
折扣因子γ	0.9
Dropout	0.1
Actor网络学习率	0.000 2
Critic网络学习率	0.000 4
软更新系数τ	0.000 01

基于深度强化学习的可见光定位通信一体化功率分配研究

Research on power allocation of integrated VLPC based on deep reinforcement learning

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 24

相关文章 15

Metrics

推荐阅读 0

[1]	马玲, 樊漆亮, 许婷, 郭冠琛, 张圣林, 孙永谦, 张玉志. 基于强化学习的在线离线混部云环境下的调度框架[J]. 通信学报, 2023, 44(6): 90-102.
[2]	金彪, 李逸康, 姚志强, 陈瑜霖, 熊金波. GenFedRL：面向深度强化学习智能体的通用联邦强化学习框架[J]. 通信学报, 2023, 44(6): 183-197.
[3]	李元诚, 秦永泰. 基于深度强化学习的软件定义安全中台QoS实时优化算法[J]. 通信学报, 2023, 44(5): 181-192.
[4]	周大成, 陈鸿昶, 何威振, 程国振, 扈红超. 基于深度强化学习的微服务多维动态防御策略研究[J]. 通信学报, 2023, 44(4): 50-63.
[5]	许国良, 谭峰, 冉泳屹, 陈丰. 面向多波束卫星系统的波束跳变与覆盖控制联合优化算法[J]. 通信学报, 2023, 44(4): 78-86.
[6]	赵越, 李赞, 李冰, 郝本建. TDoA定位盲区分析与节点部署策略研究[J]. 通信学报, 2023, 44(1): 1-13.
[7]	宋长庆, 赵宏志, 秦俪之, 邵士海. 频相失配下跳频保密通信性能分析与功率优化[J]. 通信学报, 2022, 43(9): 42-56.
[8]	许文俊, 吴思雷, 王凤玉, 林兰, 李国军, 张治. 基于多智能体强化学习的大规模灾后用户分布式覆盖优化[J]. 通信学报, 2022, 43(8): 1-16.
[9]	沙宗轩, 霍如, 孙闯, 汪硕, 黄韬. 基于深度强化学习的转发效能感知流量调度算法[J]. 通信学报, 2022, 43(8): 30-40.
[10]	张宇, 程旻. NDN中边缘计算与缓存的联合优化[J]. 通信学报, 2022, 43(8): 164-175.
[11]	孙钢灿, 吴新李, 郝万明, 朱政宇. 基于时延线阵列的毫米波NOMA系统混合预编码设计和功率分配[J]. 通信学报, 2022, 43(6): 179-188.
[12]	左珮良, 侯少龙, 郭超, 蒋华, 王文博. 基于强化学习的多层卫星网络边缘安全决策方法[J]. 通信学报, 2022, 43(6): 189-199.
[13]	黄英, 万泽含, 雷菁, 赖恪. 基于QCSK的持续噪声式隐蔽传输方案[J]. 通信学报, 2022, 43(4): 123-132.
[14]	张先超, 赵耀, 叶海军, 樊锐. 无线网络多用户干扰下智能发射功率控制算法[J]. 通信学报, 2022, 43(2): 15-21.
[15]	李传煌, 陈泱婷, 唐晶晶, 楼佳丽, 谢仁华, 方春涛, 王伟明, 陈超. QL-STCT：一种SDN链路故障智能路由收敛方法[J]. 通信学报, 2022, 43(2): 131-142.