星地融合网络中基于Q学习的切换算法研究

doi:10.11959/j.issn.1000-436x.2015240

摘要/Abstract

摘要：

基于地面辅助基站（ATC）的星地融合网络（MSS-ATC）具有覆盖范围广、用户体验佳的特点，切换机制是该融合网络主要研究的问题之一。针对卫星链路时延大、卫星网用户速度范围广的特点，综合考虑了用户接收信号强度（RSS）和用户运动速度，提出了一种基于卡尔曼滤波和Q学习的切换决策算法。比较了所提算法与传统算法在链路衰减率、切换次数和网络收益的性能，实验结果表明所提算法在性能上得到了很大的提升，并且能很好地适应高速运动状态。

关键词: 切换, MSS-ATC, RSS预测, Q学习

Abstract:

In the integrated satellite-terrestrial communication system,i.e.,mobile satellite system with ancillary terres-trial component (MSS-ATC),the long transmission delay of satellite link was a huge challenge which may lead to high handoff dropping probability.In order to address this problem,a novel handoff decision strategy was proposed based on the predictive RSS and Q-learning algorithm.Extensive simulation results demonstrate that the proposed scheme can de-crease the handoff dropping probability,reduce the unnecessary handoff times and maximize the network reward.In ad-dition,the proposed scheme can also adapt to the situation of high-speed movement very well.

Key words: handoff scheme, MSS-ATC, predictive RSS, Q-learning

熊丹妮,李屹. 星地融合网络中基于Q学习的切换算法研究[J]. 通信学报, 2015, 36(9): 252-258.

Dan-ni XIONG,Yi LI. Q-learning based handoff algorithm for satellite system with ancillary terrestrial component[J]. Journal on Communications, 2015, 36(9): 252-258.

图/表 7

图1

图2

图3

图4

图5

图6

图7

参考文献 24

[1]	CHINI P , GIAMBENE G , KOTA S . A survey on mobile satellite sys-tems[J]. International Journal of Satellite Communications and Net-working, 2010,28(1): 29-57.
[2]	AHN D S , KIM H W , AHN J , et al. Integrated/hybrid satellite and terrestrial networks for satellite imt-advanced services[J]. International Journal of Satellite Communications and Networking, 2011,29(3): 269-282.
[3]	PARSONS , SINGH R . An ATC primer:the future of communica-tions[J]. Mobile Satellite Ventures, 2006.
[4]	HALVORSON E , EISENMAN A , EDALAT F , et al. Global resource manager for mobile satellite systems with ancillary terrestrial compo-nents[A]. IEEE Sarnoff Symposium[C]. 2010.1-6.
[5]	ZOU D , MENG W , HAN S . Euclidean distance based handoff algo-rithm for fingerprint positioning of WLAN system[A]. IEEE Wireless Communications and Networking Conference (WCNC)[C]. 2013.1564-1568.
[6]	CHANG B J , CHEN J F , HSIEH C H , et al. Markov decision proc-ess-based adaptive vertical handoff with RSS prediction in heteroge-neous wireless networks[A]. IEEE Wireless Communications and Networking Conference(WCNC)[C]. 2009.1-6.
[7]	SODERMAN P , EKLUND J , GRINNEMO K J , et al. Handover in the wild:the feasibility of vertical handover in commodity smart-phones[A]. IEEE International Conference on Communications(ICC)[C]. 2013.6401-6406.
[8]	LIU S , CHANG Y , WANG G , YANG D . Vertical handoff scheme concerning mobility in the two-hierarchy network[A]. IEEE GLOBECOM Workshops (GC Wkshps)[C]. 2011.237-241.
[9]	LAHBY M , CHERKAOUI L , ADIB A . An enhanced-topsis based network selection technique for next generation wireless networks[A]. International Conference on Telecommunications[C]. 2013.1-5.
[10]	WU J S , YANG S F , HWANG B J . A terminal-controlled vertical handover decision scheme in IEEE 802.21-enabled heterogeneous wireless networks[J]. International Journal of Communication Systems, 2009,22(7): 819-834.
[11]	CHEN J , WEI Z , WANG Y , et al. A service-adaptive multi-criteria vertical handoff algorithm in heterogeneous wireless networks[A]. In-ternational Symposium on Personal Indoor and Mobile Radio Com-munications (PIMRC)[C]. 2012.899-904.
[12]	ZHANG X , LI Y , LI J , ZHAO K , ZHANG T . Proximate control stream assisted video transcoding for heterogeneous content delivery net-work[A]. IEEE International Conference on Image Processing (ICIP)[C]. 2014.2552-2555.
[13]	YIN R , CHAI R , CHEN Q . Joint utility optimization based vertical handoff algorithm in heterogeneous network[A]. IEEE Global Com-munications Conference (GLOBECOM)[C]. 2012.5243-5248.
[14]	WATKINS C , DAYAN P . Q-learning,technical note[J]. Machine Learning, 1992,8:279-292.
[15]	MARICHAMY P , CHAKRABARTI S , MASKARA S L . Performance evaluation of handoff detection schemes[A]. Conference on Conver-gent Technologies for the Asia-Pacific Region (TENCON)[C]. 2003.643-646.
[16]	RISTIC B , ARULAMPALAM S , GORDON N . Beyond the Kalman Filter:Particle Filters for Tracking Applications[M]. Artech House, 2004.
[17]	HARVEY , ANDREW C . Forecasting,Structural Time Series Models and the Kalman Filter[M]. Cambridge University Press, 1990.
[18]	VEERAVALLI V V , KELLY O E . A locally optimal handoff algorithm for cellular communications[J]. IEEE Transactions on Vehicular Technology, 1997,146(3): 603-609.
[19]	SADEK M , ISSA S A . Handoff algorithm for mobile satellite systems with ancillary terrestrial component[A]. IEEE International Confer-ence on Communications (ICC)[C]. 2012.2763-2767.
[20]	LIOLIS K P , OMEZ-VILARDEB O J G , CASINI E , et al. Statistical modeling of dual-polarized mimo land mobile satellite channels[J]. IEEE Transactions on Communications, 2010,58(11): 3077-3083.
[21]	CORAZZA G E , VATALARO F . A statistical model for land mobile satellite channels and its application to nongeostationary orbit sys-tems[J]. IEEE Transactions on Vehicular Technology, 1994,43(3): 738-742.
[22]	TABRIZI H , FARHADI G , CIOFFI J . Dynamic handoff decision in heterogeneous wireless systems:Q-learning approach[A]. IEEE Inter-national Conference on Communications (ICC)[C]. 2012.3217-3222.
[23]	NIE J , HAYKIN S . A dynamic channel assignment policy through Q-learning[J]. IEEE Transactions on Neural Networks, 1999,10(6): 1443-1455.
[24]	POLLINI G P . Trends in handover design[J]. IEEE Communications Magazine, 1996,34(3): 82-90.