基于强化学习的异构无线网络资源管理算法

doi:10.11959/j.issn.1000-0801.2015195

Abstract

Abstract:

In order to make full use of the resources of all kinds of wireless network，the integration of heterogeneous network is necessary.However，when it comes to the heterogeneous network integration，the problems of call request access control and resource management emerge.A reinforcement-learning-based algorithm was presented for heterogeneous wireless network resource management.D2D （device-to-device）communication was introduced into the proposed algorithm and the appropriate network for access could be selected according to different traffic types，terminal mobility，network load status and so on.Meanwhile，to reduce the storage requirement，the neural network technology was used to solve the problem of continuous state space.Simulation results show that the proposed algorithm has an efficient learning ability to achieve autonomous radio resource management，which effectively improves the spectrum utility and reduces the blocking probability.

Key words: heterogeneous wireless network, access control, resource management, reinforcement learning, Q-learning

Chenwei Feng,Jiangnan Yuan. Heterogeneous Wireless Network Resource Management Algorithm Based on Reinforcement Learning[J]. Telecommunications Science, 2015, 31(8): 99-106.

Figures/Tables 12

参数		LTE	small cell	D2D
容量/（kbit·s^-1）		1 000	2 000	4 000
业务匹配系数η_V	语音	5	1	8d
	数据	1	5	8d
移动性匹配系数η_M	移动	5	1	0
	静止	1	5	5d
其他参数		折扣因子γ=0.8
		初始学习率 α₀=0.5
		初始探索概率 ε₀=0.5
		$d = {\begin{matrix} 1, & 终端具有 D 2 D 模式 \\ 0, & 其他 \end{matrix}$

References 23

1	Luo J ， Mukerjee R ， Dillinger M ，et al. Investigation of radio resource scheduling in WLANs coupled with 3G cellular network. IEEE Communications Magazine， 2003，41（6）： 108～115
2	3GPP TR 25.881 v5.0.0.Improvement of RRM across RNS and RNS/BSS（Release5）. ， 2001
3	Song Q ， Jamalipour A . Network selection in an integrated wireless LAN and UMTS environment using mathematical modeling and computing techniques. IEEE Wireless Communications， 2005，12（3）： 42～48
4	Zhang Y ， Chen J ， Zhang P . Autonomic joint radio resource management in B3G environment using reinforcement learning. Proceedings of the 6th Annual Wireless Telecommunications Symposium（WTS 2007）， Pomona，California，USA， 2007
5	Barto A G . Reinforcement Learning：an Introduction. Cambridge： MIT Press, 1998
6	Kaelbling L P ， Littman M L ， Moore A W . Reinforcement learning：a survey. Journal of Artificial Intelligence Research， 1996（4）： 237～285
7	Nie J ， Haykin S . A Q-learning-based dynamic channel assignment technique for mobile communication systems. IEEE Transactions on Vehicular Technology， 1999，48（5）： 1676～1687
8	Senouci S M ， Beylot A L ， Pujolle G . Call admission control in cellular networks：a reinforcement learning solution. International Journal of Network Management， 2004，14（2）： 89～103
9	Haddad M ， Altman Z ， Elayoubi S E ，et al. A nash-stackelberg fuzzy Q-learning decision approach in heterogeneous cognitive networks. Proceedings of Global Telecommunications Conference （GLOBECOM 2010）， Miami，Florida，USA， 2010： 1～6
10	Simsek M ， Czylwik A . Decentralized Q-learning of LTE-femtocells for interference reduction in heterogeneous networks using cooperation. Proceedings of 2012 International ITG Workshop on Smart Antennas（WSA）， Dresden，Germany， 2012： 86～91
11	Saker L ， Ben Jemaa S ， Elayoubi S E . Q-learning for joint access decision in heterogeneous networks. Proceedings of Wireless Communications and Networking Conference， Budapest，Hungary， 2009： 1～5
12	Tabrizi H ， Farhadi G ， Cioffi J . Dynamic handoff decision in heterogeneous wireless systems：Q-learning approach. Proceedings of 2012 IEEE International Conference on Communications （ICC）， Ottawa，Canada， 2012： 3217～3222
13	张永靖，冯志勇，张平 . 基于Q学习的自主联合无线资源管理算法. 电子与信息学报， 2008，30（3）： 676～680 Zhang Y ， Feng Z ， Zhang P . A Q-learning based autonomic join radio resource management algorithm. Journal of Electronics and Information Technology， 2008，33（3）： 676～680
14	Zhao Y ， Zhou W ， Zhu Q . Q-learning based heterogeneous network selection algorithm. Proceedings of the 2nd World Congress on Computer Science and Information Engineering， Changchun，China， 2011： 471～477
15	Chen Y H ， Chang C J ， Huang C Y . Fuzzy Q-learning admission control for WCDMA/WLAN heterogeneous networks with multimedia traffic. IEEE Transactions on Mobile Computing， 2009，8（11）： 1469～1479
16	谭力，陈亚迷，李一喆等. 一种基于 Q-学习的认知无线网络接入方案. 新型工业化， 2011（9）： 37～44 Tan L ， Chen Y ， Li Y ，et al. Q-learning based cognitive wireless network access mechanism. Journal of New Industrialization， 2011（9）： 37～44
17	陈旋 . 基于Q学习的LTE/WLAN网络接入控制算法研究（硕士学位论文）. 哈尔滨工业大学， 2013 Chen X . Call admission control in LTE/WLAN network based on Q-learning（master dissertation）. Harbin Institute of Technology， 2013
18	Tan X ， Luan X ， Cheng Y ，et al. Cell selection in two-tier femtocell networks using Q-learning algorithm. Proceedings of the 16th International Conference on Advanced Communication Technology， Pyeongchang，Korea， 2014： 1031～1035
19	Doppler K ， Rinne M ， Wijting C ，et al. Device-to-device communication as an underlay to LTE-Advanced networks. IEEE Communications Magazine， 2009，47（12）： 42～49
20	Barto A G ， Bradtke S J ， Singh S P . Learning to act using real-time dynamic programming. Artificial Intelligence， 1995，72（1）： 81～138
21	Watkins C J C H ， Dayan P . Q-learning. Machine Learning， 1992，8（3～4）： 279～292
22	叶培智 . 异构无线网络接入选择算法的研究（硕士学位论文）. 厦门大学， 2013 Ye P Z . Research on access selection in heterogeneous wireless networks（master dissertation）. Xiamen University， 2013
23	Senouci S ， Beylot A ， Pujolle G Call admission control in cellular networks：a reinforcement learning solution. International Journal of Network Management， 2004，14（2）： 89～103

Metrics

Recommended 0

No Suggested Reading articles found!

网络	RRA	LBJA	QLJA
LTE	0.04	0.05	0.38
small cell	0.03	-0.05	-0.35

网络	RRA	LBJA	QLJA
LTE	0.03	0.07	-0.39
small cell	-0.04	-0.02	0.38

[1]	Xuerong WANG, Zhengzhi TANG, Yinchuan LI, Meiyu QI, Jianbo ZHU, Liang ZHANG. Delay-sensitive traffic intellisense scheduling based on optimal decision tree [J]. Telecommunications Science, 2023, 39(4): 120-132.
[2]	Xiwen LIAO, Supeng LENG, Yujun MING, Tianyang LI. Digital twin based intelligent urban traffic forecasting and guidance strategy [J]. Telecommunications Science, 2023, 39(3): 70-79.
[3]	Yu KANG, Yaqiong LIU, Tongyu ZHAO, Guochu SHOU. A survey on AI algorithms applied in communication and computation in Internet of vehicles [J]. Telecommunications Science, 2023, 39(1): 1-19.
[4]	Yu DING, Chenkai LI, Huimei HAN, Weidang LU, Yuanhong REN, Yuan GAO, Jiang CAO. Multi-agent heterogeneous network selection method based on 5G UAV communication [J]. Telecommunications Science, 2022, 38(8): 28-36.
[5]	Xin LI, Jun SUN. Value-difference learning based mMTC devices access algorithm in multi-cell network [J]. Telecommunications Science, 2022, 38(6): 82-90.
[6]	Nan JIN, Ruiqin WANG, Yuecong LU. Ebbinghaus forgetting curve and attention mechanism based recommendation algorithm [J]. Telecommunications Science, 2022, 38(10): 89-97.
[7]	Qian SUN, Lin TIAN, Yiqing ZHOU, Chen FENG, Yuanyuan WANG, Jihua ZHOU. Resource management in sliced networks with mobile edge computing for B5G and 6G [J]. Telecommunications Science, 2021, 37(6): 55-65.
[8]	Jianwu ZHANG, Luxin WANG, Lingfen SUN, Qianye ZHANG, Hangguan SHAN. An survey on application of artificial intelligence in 5G system [J]. Telecommunications Science, 2021, 37(5): 14-31.
[9]	Zhongli WU, Yuanyuan CAO, Wenrui HUANG, Bin DAI, Yijun MO. On-demand intelligent routing technology for deterministic network [J]. Telecommunications Science, 2021, 37(11): 11-16.
[10]	Yunhe YU, Jun SUN. Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication [J]. Telecommunications Science, 2021, 37(11): 41-50.
[11]	Wei DING. Network security threat prevention and control system of electric power monitoring systems for wind farm [J]. Telecommunications Science, 2020, 36(5): 138-144.
[12]	Guofeng HE. Application protection in 5G cloud network using zero trust architecture [J]. Telecommunications Science, 2020, 36(12): 123-132.
[13]	Fei GUI,Yang CHENG,Dan LI,Sihong HONG. Internet intelligent routing architecture and algorithm [J]. Telecommunications Science, 2020, 36(10): 12-20.
[14]	Shanshan TU,Jinliang YU,Yuan MENG,M WWAQAS,Lei LIU. Secure relay node selection method based on Q-learning for fog computing in 5G network [J]. Telecommunications Science, 2019, 35(7): 60-68.
[15]	Mingjie HE,Jie ZHANG,Shiguang SHAN. A survey of neural architecture search [J]. Telecommunications Science, 2019, 35(5): 43-50.

Heterogeneous Wireless Network Resource Management Algorithm Based on Reinforcement Learning

RichHTML

PDF下载

Knowledge

Cited

Abstract

Cite this article

share this article

Figures/Tables 12

References 23

Related Articles 15

Metrics

Recommended 0