基于深度强化学习的OFDM自适应导频设计

doi:10.11959/j.issn.1000-436x.2023169

Abstract

Abstract:

For orthogonal frequency division multiplexing (OFDM) systems, an adaptive pilot design algorithm based on deep reinforcement learning was proposed.The pilot design problem was formulated as a Markov decision process, where the index of pilot positions was defined as actions.A reward function based on mean squared error (MSE) reduction strategy was formulated, and deep reinforcement learning was employed to update the pilot positions.The pilot was adaptively and dynamically allocated based on channel conditions, thereby utilizing channel characteristics to combat channel fading.The simulation results show that the proposed algorithm has significantly improved channel estimation performance compared with the traditional pilot uniform allocation scheme under three typical multipath channels of 3GPP.

Key words: OFDM, deep reinforcement learning, Markov decision process, multipath channel

CLC Number:

TN92

Qiaoshou LIU, Xiong ZHOU, Shuang LIU, Yifeng DENG. Adaptive pilot design for OFDM based on deep reinforcement learning[J]. Journal on Communications, 2023, 44(9): 104-114.

Figures/Tables 12

References 19

[1]	SOLTANI M , POURAHMADI V , SHEIKHZADEH H . Pilot pattern design for deep learning-based channel estimation in OFDM systems[J]. IEEE Wireless Communications Letters, 2020,9(12): 2173-2176.
[2]	MASHHADI M B , GüNDüZ D . Pruning the pilots:deep learning-based pilot design and channel estimation for MIMO-OFDM systems[J]. IEEE Transactions on Wireless Communications, 2021,20(10): 6315-6328.
[3]	CHEN H , ZHANG Q Q , LONG R Z ,et al. Pilot design and signal detection for symbiotic radio over OFDM carriers[C]// Proceedings of IEEE Global Communications Conference. Piscataway:IEEE Press, 2023: 1887-1892.
[4]	CAO J , ZHU X , JIANG Y F ,et al. Independent pilots versus shared pilots:short frame structure optimization for heterogeneous-traffic URLLC networks[J]. IEEE Transactions on Wireless Communications, 2022,21(8): 5755-5769.
[5]	LIN X , LIU A J , HAN C ,et al. Joint pilot spacing and power optimization scheme for nonstationary wireless channel:a deep reinforcement learning approach[J]. IEEE Wireless Communications Letters, 2023,12(3): 540-544.
[6]	RAO R M , MAROJEVIC V , REED J H . Adaptive pilot patterns for CA-OFDM systems in nonstationary wireless channels[J]. IEEE Transactions on Vehicular Technology, 2018,67(2): 1231-1244.
[7]	LI F , SHEN B W , GUO J L ,et al. Dynamic spectrum access for Internet-of-things based on federated deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2022,71(7): 7952-7956.
[8]	CHEN M , LIU A , LIU W ,et al. RDRL:a recurrent deep reinforcement learning scheme for dynamic spectrum access in reconfigurable wireless networks[J]. IEEE Transactions on Network Science and Engineering, 2021,9(2): 364-376.
[9]	HAN H , XU Y , JIN z ,et al. primary-user-friendly dynamic spectrum anti-jamming access:a GAN-enhanced deep reinforcement learning Approach[J]. IEEE Wireless Communications Letters, 2021,11(2): 258-262.
[10]	LI J , GAO H , LV T ,et al. Deep reinforcement learning based computation offloading and resource allocation for MEC[C]// Proceedings of IEEE Wireless Communications and Networking Conference (WCNC). Piscataway:IEEE Press, 2018: 1-6.
[11]	YANG H , WEI Z , FENG Z ,et al. Intelligent computation offloading for MEC-based cooperative vehicle infrastructure system:a deep reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2022,71(7): 7665-7679.
[12]	WANG J , ZHAO L , LIU J ,et al. Smart resource allocation for mobile edge computing:a deep reinforcement learning approach[J]. IEEE Transactions on Emerging Topics in Computing, 2019,9(3): 1529-1541.
[13]	NGUYEN T T , REDDI V J . Deep reinforcement learning for cyber security[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023,34(8): 3779-3795.
[14]	ZHANG Y , MOU Z , GAO F ,et al. UAV-enabled secure communications by multi-agent deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2020,69(10): 11599-11611.
[15]	OH M S , HOSSEINALIPOUR S , KIM T ,et al. Channel estimation via successive denoising in MIMO OFDM systems:a reinforcement learning approach[C]// Proceedings of IEEE International Conference on Communications (ICC). Piscataway:IEEE Press, 2021: 1-6.
[16]	CHU M , LIU A , LAU V K N ,et al. Deep reinforcement learning based end-to-end multiuser channel prediction and beamforming[J]. IEEE Transactions on Wireless Communications, 2022,21(12): 10271-10285.
[17]	HUANG R , WONG V W S , SCHOBER R . Throughput optimization for grant-free multiple access with multiagent deep reinforcement learning[J]. IEEE Transactions on Wireless Communications, 2020,20(1): 228-242.
[18]	WANG Z , SCHAUL T , HESSEL M ,et al. Dueling network architectures for deep reinforcement learning[C]// Proceedings of International Conference on Machine Learning. New York:PMLR, 2016: 1995-2003.
[19]	European Telecommunications Standards Institute. 3GPP:TS36.104[S].[2023-05-06].

Metrics

Recommended 0

No Suggested Reading articles found!

参数		值
网络层	F₁	Linear,840
	F₂	Linear,64,ReLU
	F₃	Linear,128,ReLU
	S₁	Linear,128,ReLU
	S₂	Linear,1
	A₁	Linear,128,ReLU
	A₂	Linear,54070
batch-size		256
经验回放池C		5 000
W		5 000
ε		0.9
γ		1
η		0.01
优化器		Adam

信道	时延/ns	功率增益/dB
	0	0.0
	30	-1.0
	70	-2.0
	90	-3.0
EPA	110	-8.0
	190	-17.2
	410	-20.8
	—	—
	—	—
	0	0.0
	30	-1.5
	150	-1.4
	310	-3.6
EVA	370	-0.6
	710	-9.1
	1 090	-7.0
	1 730	-12.0
	2 510	-16.*
	0	-1.0
	50	-1.0
	120	-1.0
	200	0.0
ETU	230	0.0
	500	0.0
	1 600	-3.0
	2 300	-5.0
	5 000	-7.0
	0	-1.0

Adaptive pilot design for OFDM based on deep reinforcement learning

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 19

Related Articles 15

Metrics

Recommended 0

[1]	Runzi LIU, Tianci MA, Weihua WU, Chenhong YAO, Qinghai YANG. Dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning [J]. Journal on Communications, 2023, 44(7): 207-217.
[2]	Ningjiang CHEN, Linming LIAN, Pingjie OU, Xuemei YUAN. D2D cooperative caching strategy based on graph collaborative filtering model [J]. Journal on Communications, 2023, 44(7): 136-148.
[3]	Feng ZENG, Zheng ZHANG, Zhigang CHEN. Computation offloading and resource allocation strategy based on deep reinforcement learning [J]. Journal on Communications, 2023, 44(7): 124-135.
[4]	Biao JIN, Yikang LI, Zhiqiang YAO, Yulin CHEN, Jinbo XIONG. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents [J]. Journal on Communications, 2023, 44(6): 183-197.
[5]	Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform [J]. Journal on Communications, 2023, 44(5): 181-192.
[6]	Guang YANG, Zhaoyang WU, Min NIE, Xiaohong YAN, Fan JIANG. Research on PAPR reduction algorithm based on CWGAN-SLM for multi-wavelet OFDM system [J]. Journal on Communications, 2023, 44(4): 99-110.
[7]	Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system [J]. Journal on Communications, 2023, 44(4): 78-86.
[8]	Yu ZHANG, Min CHENG. Joint optimization of edge computing and caching in NDN [J]. Journal on Communications, 2022, 43(8): 164-175.
[9]	Zongxuan SHA, Ru HUO, Chuang SUN, Shuo WANG, Tao HUANG. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 30-40.
[10]	Xianchao ZHANG, Yao ZHAO, Haijun YE, Rui FAN. Intelligent transmit power control algorithm for the multi-user interference of wireless network [J]. Journal on Communications, 2022, 43(2): 15-21.
[11]	Xin SU, Leilei MENG, Yiqing ZHOU, Wu CELIMUGE. Maritime mobile edge computing offloading method based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(10): 133-145.
[12]	Linsen XU, Wei YANG, Hongxian TIAN. Channel interleaver design and performance analysis for ultrasonic through-metal communication [J]. Journal on Communications, 2022, 43(10): 1-11.
[13]	Kai MEI, Haitao ZHAO, Xiaoran LIU, Jun LIU, Jun XIONG, Baoquan REN, Jibo WEI. Efficient model-and-data based channel estimation algorithm [J]. Journal on Communications, 2022, 43(1): 59-70.
[14]	Li’na DU, Li ZHUO, Shuo YANG, Jiafeng LI, Jing ZHANG. Survey on reinforcement learning based adaptive bit rate algorithm for mobile video streaming services [J]. Journal on Communications, 2021, 42(9): 205-217.
[15]	Guo LI, Ni WEN, Fengkui GONG, Jian ZHANG, Sihan ZHANG. Interference detection and diversity suppression algorithm of OFDM system under partial-band jamming [J]. Journal on Communications, 2021, 42(9): 194-204.