无线网络中基于深度Q学习的传输调度方案

doi:10.11959/j.issn.1000-436x.2018058

Abstract

Abstract:

To cope with the problem of data transmission in wireless networks,a deep Q learning based transmission scheduling scheme was proposed.The Markov decision process system model was formulated to describe the state transition of the system.The Q learning algorithm was adopted to learn and explore the system states transition information in the case of unknown system states transition probability to obtain the approximate optimal strategy of the schedule node.In addition,when the system state scale was big,the deep learning method was employed to map the relation between state and behavior to solve the problem of the large amount of computation and storage space in Q learning process.The simulation results show that the proposed scheme can approach the optimal strategy based on strategy iteration in terms of power consumption,throughput,packets loss rate.And the proposed scheme has a lower complexity,which can solve the problem of the curse of dimensionality.

Key words: wireless network transmission, Markov decision process, Q learning, deep learning

CLC Number:

TN929.5

Jiang ZHU,Tingting WANG,Yonghui SONG,Yali LIU. Transmission scheduling scheme based on deep Q learning in wireless network[J]. Journal on Communications, 2018, 39(4): 35-44.

Figures/Tables 10

参数	值描述
信噪比门限/dB	snr=[?6.28,?1.28,1.28]
多普勒频移/Hz	f_D=50
帧长/s	T_f=2 ×10^?3
时隙数1	I₁=5×10³
时隙数2	I₂=1×10³
噪声功率	1×10^?3
缓存区压力系数	θ=0.5
到达率	λ=[0.1,…,0.9]
误比特率约束	BER=10^?3
折扣系数	γ=0.9
索引权重	$ς = \frac{1}{\sqrt{2}}$
MQL学习因子	α∈ (0,1]
SAE各层神经元数	9,15,15,12]
权值权重	μ=3×10^?3
SAE学习速率	1×10^?2
训练误差精度	error=1×10^?5

References 21

[17]	KOBAYASHI T , SHIBUYA T , TANAKA F ,et al. Q-learning in continuous state-action space by using a selective desensitization neural network[J]. IEICE Technical Report Neurocomputing, 2011,111: 119-123.
[18]	周文云 . 强化学习维数灾问题解决方法研究[D]. 苏州:苏州大学, 2009.
[1]	朱江, 徐斌阳, 李少谦 . 一种基于马尔可夫决策过程的认知无线电网络传输调度方案[J]. 电子与信息学报, 2009,31(8): 2019-2023.
	ZHU J , XU B Y , LI S Q . A transmission and scheduling scheme based on Markov decision process in cognitive radio networks[J]. Journal of Electronics ＆ Information Technology, 2009,31(8): 2019-2023.
[18]	ZHOU W Y . Research on the curse of dimensionality in reinforcement learning[D]. Suzhou:Soochow University, 2009.
[19]	LIU W , LIU N , SUN H ,et al. Dispatching algorithm design for elevator group control system with Q-learning based on a recurrent neural network[C]// Control and Decision Conference. 2013: 3397-3402.
[2]	ZHU J , PENG Z Z , LI F . A transmission and scheduling scheme based on W-learning algorithm in wireless networks[C]// 8th International ICST Conference on Communications and Networking in China (CHINACOM). 2013: 85-90.
[3]	LI H , HAN Z . Competitive spectrum access in cognitive radio networks:graphical game and learning[C]// Wireless Communications and Networking Conference (WCNC). 2010: 1-6.
[20]	WEI Q , LEWISF L , SUN Q ,et al. Discrete-time deterministic Q-learning:a novel convergence analysis[J]. IEEE transactions on cybernetics, 2016: 1-14.
[21]	李军, 徐玖平 . 运筹学:非线性系统优化[M]. 北京: 科学出版社, 2003.
[4]	林晓辉, 谭宇, 张俊玲 ,等. 无线传输中基于马尔可夫决策的高能效策略[J]. 系统工程与电子技术, 2014,36(7): 1433-1438.
	LIN X H , TAN Y , ZHANG J L ,et al. MDP-based energy efficient policy for wireless transmission[J]. Systems Engineering and Electronics, 2014,36(7): 1433-1438.
[5]	WANG H S , MOAYERI N . Finite-state Markov channel-a useful model for radio communication channels[J]. IEEE Transactions on Vehicular Technology, 1995,44(1): 163-171.
[6]	GAO Q , ZHU G , LIN S ,et al. Robust QoS-aware cross-layer design of adaptive modulation transmission on OFDM systems in high-speed railway[J]. IEEE Access, 2016,PP(99):1.
[7]	CHEN X , CHEN W . Delay-optimal probabilistic scheduling for low-complexity wireless links with fixed modulation and coding:a cross-layer design[J]. IEEE Transactions on Vehicular Technology, 2016:1.
[8]	LAU V K N , . Performance of variable rate bit interleaved coding for high bandwidth efficiency[C]// The Vehicular Technology Conference. 2000: 2054-2058.
[9]	CHUNG S T , GOLDSMITH A J . Degrees of freedom in adaptive modulation:a unified view[C]// IEEE Transactions on Communications. 2001: 1561-1571.
[10]	WEI Q , LIU D , SHI G . A novel dual iterative Q-learning method for optimal battery management in smart residential environments[J]. IEEE Transactions on Industrial Electronics, 2015,62(4): 2509-2518.
[11]	NI J , LIU M , REN L ,et al. A multiagent Q-learning-based optimal allocation approach for urban water resource management system[J]. IEEE Transactions on Automation Science ＆ Engineering, 2014,11(1): 204-214.
[12]	SILVER D , HUANG A , MADDISON C J ,et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016,529(7587): 484-489.
[13]	WEI C , ZHANG Z , QIAO W ,et al. An adaptive network-based reinforcement learning method for MPPT control of PMSG wind energy conversion systems[J]. IEEE Transactions on Power Electronics, 2016:1.
[14]	KIM T , SUN Z , COOK C ,et al. Invited-cross-layer modeling and optimization for electromigration induced reliability[C]// Design Automation Conference. 2016: 1-6.
[15]	COMSA I S , ZHANG S , AYDIN M . A novel dynamic Q-learning-based scheduler technique for LTE-advanced technologies using neural networks[C]// Conference on Local Computer Networks. 2012: 332-335.
[16]	TENG T H , TAN A H . Fast reinforcement learning under uncertainties with self-organizing neural networks[C]// IEEE / WIC / ACM International Conference on Web Intelligence and Intelligent Agent Technology. 2015: 51-58.
[21]	LI J , XU J P . Operations research:nonlinear system optimization[M]. Beijing: Science PressPress, 2003.

Metrics

Recommended 0

No Suggested Reading articles found!

算法	是否依赖先验信息	指数运算	乘除运算	加减运算	比较运算
SI法	是	0	D+S	D	D
本文算法	否	2A	4A	2A	2A
W学习法	否	2A	3A	3A	A
RS法	否	0	1	0	0

Transmission scheduling scheme based on deep Q learning in wireless network

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 21

Related Articles 15

Metrics

Recommended 0

[1]	Dongyu CHEN, Hua CHEN, Limin FAN, Yifang FU, Jian WANG. Research on test strategy for randomness based on deep learning [J]. Journal on Communications, 2023, 44(6): 23-33.
[2]	Rongpeng LI, Bingyan WANG, Honggang ZHANG, Zhifeng ZHAO. Design of knowledge enhanced semantic communication receiver [J]. Journal on Communications, 2023, 44(6): 70-76.
[3]	Shuai MA, Ke PEI, Huayan QI, Hang LI, Wen CAO, Hongmei WANG, Hailiang XIONG, Shiyin LI. Research on geomagnetic indoor high-precision positioning algorithm based on generative model [J]. Journal on Communications, 2023, 44(6): 211-222.
[4]	Jie YANG, Biao DONG, Xue FU, Yu WANG, Guan GUI. Lightweight decentralized learning-based automatic modulation classification method [J]. Journal on Communications, 2022, 43(7): 134-142.
[5]	Xiuzhang YANG, Guojun PENG, Zichuan LI, Yangqi LYU, Side LIU, Chenguang LI. Research on entity recognition and alignment of APT attack based on Bert and BiLSTM-CRF [J]. Journal on Communications, 2022, 43(6): 58-70.
[6]	Yong LIAO, Shiyi WANG. CSI feedback algorithm based on RM-Net for massive MIMO systems in high-speed mobile environment [J]. Journal on Communications, 2022, 43(5): 166-176.
[7]	Yurong LIAO, Haining WANG, Cunbao LIN, Yang LI, Yuqiang FANG, Shuyan NI. Research progress of deep learning-based object detection of optical remote sensing image [J]. Journal on Communications, 2022, 43(5): 190-203.
[8]	Zenghua ZHAO, Yuefan TONG, Jiayang CUI. Device-independent Wi-Fi fingerprinting indoor localization model based on domain adaptation [J]. Journal on Communications, 2022, 43(4): 143-153.
[9]	Yong LIAO, Gang CHENG, Yujie LI. CSI feedback algorithm based on deep unfolding for massive MIMO systems [J]. Journal on Communications, 2022, 43(12): 77-88.
[10]	Xueyuan DUAN, Yu FU, Kun WANG, Bin LI. LDoS attack detection method based on simple statistical features [J]. Journal on Communications, 2022, 43(11): 53-64.
[11]	Junyan HUO, Ruipeng QIU, Yanzhuo MA, Fuzheng YANG. Reference frame list optimization algorithm in video coding by quality enhancement of the nearest picture [J]. Journal on Communications, 2022, 43(11): 136-147.
[12]	Haiyan KANG, Yuanrui JI. Research on federated learning approach based on local differential privacy [J]. Journal on Communications, 2022, 43(10): 94-105.
[13]	Hongxia ZHANG, Qi WANG, Dengyue WANG, Ben WANG. Honeypot contract detection of blockchain based on deep learning [J]. Journal on Communications, 2022, 43(1): 194-202.
[14]	Yan YAN, Yiming CONG, Mahmood Adnan, Quanzheng SHENG. Statistics release and privacy protection method of location big data based on deep learning [J]. Journal on Communications, 2022, 43(1): 203-216.
[15]	Ye ZHU, Yilin YU, Yingchun GUO. HRDA-Net: image multiple manipulation detection and location algorithm in real scene [J]. Journal on Communications, 2022, 43(1): 217-226.