基于强化学习的IEEE 802.15.4网络区分服务策略

doi:10.11959/j.issn.1000-436x.2015149

摘要/Abstract

摘要：

为了弥补IEEE 802.15.4协议原有区分服务机制的不足，提出了一种基于BCS(backoff counter scheme)与强化学习的区分服务策略。从终端节点出发，在原优先级区分服务策略的基础上增加 BCS 退避策略以解决流量较大场合业务区分问题；针对协调器节点，提出了基于强化学习的占空比调整策略，该策略能根据不同应用需求和环境变化自适应调整占空比。仿真结果表明，提出算法能针对不同环境满足高优先级业务性能需求，并能根据流量变化进行占空比调整，具有极强环境适应性。

关键词: IEEE802.15.4/LR-WPAN, 区分服务, 退避机制, 强化学习, 占空比

Abstract:

To provide better support in differentiated service for IEEE 802.15.4,a novel differentiated service mechanism was proposed based on BCS(back off counter scheme)and reinforcement learning.In terms of end-device,BCS backoff strategy was added to original priority-based differentiated strategy to solve the service differentiation problem under higher traffic condition.Whil “ e for the coordinator,a reinforcement learning based duty-cycle adjustment algorithm was proposed toself-learning”an optimal duty-cycle according to different application requirements and environmental changes.Simulation shows that the proposed algorithm can meet the performance requirements of high-priority service under different environments and adjust the duty-cycle when traffic is changed,which showed a strong environmental adaptability.

Key words: IEEE 802.15.4/LR-WPAN, differentiated-service, BCS, reinforcement-learning, duty-cycle

钱亮,钱志鸿,李天平,全薇. 基于强化学习的IEEE 802.15.4网络区分服务策略[J]. 通信学报, 2015, 36(8): 171-181.

Liang QIAN,Zhi-hong QIAN,Tian-ping LI,Wei QUAN. IEEE 802.15.4 differentiated service strategy based on reinforcement-learning[J]. Journal on Communications, 2015, 36(8): 171-181.

图/表 18

图1

图2

图3

图4

图5

图6

图7

表1

图8

表2

图9

图10

图11

图12

图13

图14

图15

图16

参考文献 13

[1]	CALLAWAY E , GORDAY P , HESTER L . Home networking with IEEE 802.15.4:a developing standard for low-rate wireless personal area networks[J]. IEEE Communications Magazine, 2002,40(8): 70-77.
[2]	OTTO C,MILENKOVI? A , SANDERS C . System architecture of a wireless body area sensor network for ubiquitous health monitoring[J]. Journal of Mobile Multimedia, 2006,1(4): 307-326.
[3]	CAO X , CHEN J , ZHANG Y . Development of an integrated wireless sensor network micro-environmental monitoring system[J]. ISA Transactions, 2008,47(3): 247-255.
[4]	IEEE 802.15.4 Standard(2003)Part 15.4:Wireless Medium Access Control(MAC)and Physical Layer(PHY)Specifications for Low-Rate Wireless Personal Area Networks(LR-WPANs)[S]. IEEE-SA Standards Board, 2003.
[5]	TSENG P , FENG K , LIN Y . Wireless location tracking algorithms for environments with insufficient signal sources[J]. IEEE Transactions on Mobile Computing, 2009,8(12): 1676-1689.
[6]	HO C , LIN C , HWANG W . Dynamic GTS allocation scheme in IEEE 802.15.4 by multi-factor[A]. 2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing(IIH-MSP)[C]. Piraeus, 2012. 457-460.
[7]	KOUB?A A , ALVES M , NEFZI B . Improving the IEEE 802.15.4 slotted CSMA/CA MAC for time-critical events in wireless sensor networks[A]. Proceedings of the Workshop on Real Time Networks(RTN ’06)[C]. Dresden, 2006. 270-277.
[8]	KIPINS D , WILLIG A , HAUER J . The angel IEEE 802.15.4enhancement layer:coupling priority queueing and service differentiation[A]. Wireless Conference,EW 2008.14th European[C]. Prague, 2008. 1-7.
[9]	ROHM D , GOYAL M , HOSSEINI S. . A simulation based analysis of the impact of IEEE 802.15.4 MAC parameters on the performance under different traffic loads[J]. Mobile Information Systems, 2009,5(1): 81-99.
[10]	SUTTON R , BARTO A . Reinforcement Learning:An Introduction[M]. Massachusetts: MIT Press, 1998.
[11]	SUTTON R , BARTO A . Introduction to Reinforcement Learning[M]. Massachusetts: MIT Press, 1998.
[12]	Multi-armed bandit[EB/OL]. . 2014.
[13]	HERNANDEZ-GARDIOL N , MAHADEVAN S . Hierarchical memory-based reinforcement learning[A]. 14th Annual Neural Information Processing Systems Conference[C]. 2001. 1047-1053.

二进制表示	队列占用率/%
000	0～12.5
001	12.5～25
010	25～37.5
011	37.5～50
100	50～62.5
101	62.5～75
110	75～87.5
111	87.5～100

参数		值
业务量大小/(packet·s^?1)		0.2
数据分组长度/byte		70
节点个数		7
BO(SO_max)		7
仿真时间/s		8 000
	initialEnergy	1 000
	rxPower	35.28×10^?3
能量消耗/mW	txPower	31.32×10^?3
	idlePower	712×10^?6
	sleepPower	144×10^?9
强化学习	Learning-RateExploring-Rate	0.10.1

[1]	马玲, 樊漆亮, 许婷, 郭冠琛, 张圣林, 孙永谦, 张玉志. 基于强化学习的在线离线混部云环境下的调度框架[J]. 通信学报, 2023, 44(6): 90-102.
[2]	金彪, 李逸康, 姚志强, 陈瑜霖, 熊金波. GenFedRL：面向深度强化学习智能体的通用联邦强化学习框架[J]. 通信学报, 2023, 44(6): 183-197.
[3]	李元诚, 秦永泰. 基于深度强化学习的软件定义安全中台QoS实时优化算法[J]. 通信学报, 2023, 44(5): 181-192.
[4]	周大成, 陈鸿昶, 何威振, 程国振, 扈红超. 基于深度强化学习的微服务多维动态防御策略研究[J]. 通信学报, 2023, 44(4): 50-63.
[5]	许国良, 谭峰, 冉泳屹, 陈丰. 面向多波束卫星系统的波束跳变与覆盖控制联合优化算法[J]. 通信学报, 2023, 44(4): 78-86.
[6]	许文俊, 吴思雷, 王凤玉, 林兰, 李国军, 张治. 基于多智能体强化学习的大规模灾后用户分布式覆盖优化[J]. 通信学报, 2022, 43(8): 1-16.
[7]	沙宗轩, 霍如, 孙闯, 汪硕, 黄韬. 基于深度强化学习的转发效能感知流量调度算法[J]. 通信学报, 2022, 43(8): 30-40.
[8]	马帅, 李兵, 盛海鸿, 谷荣妍, 周辉, 王洪梅, 王悦, 李世银. 基于深度强化学习的可见光定位通信一体化功率分配研究[J]. 通信学报, 2022, 43(8): 121-130.
[9]	张宇, 程旻. NDN中边缘计算与缓存的联合优化[J]. 通信学报, 2022, 43(8): 164-175.
[10]	左珮良, 侯少龙, 郭超, 蒋华, 王文博. 基于强化学习的多层卫星网络边缘安全决策方法[J]. 通信学报, 2022, 43(6): 189-199.
[11]	张先超, 赵耀, 叶海军, 樊锐. 无线网络多用户干扰下智能发射功率控制算法[J]. 通信学报, 2022, 43(2): 15-21.
[12]	李传煌, 陈泱婷, 唐晶晶, 楼佳丽, 谢仁华, 方春涛, 王伟明, 陈超. QL-STCT：一种SDN链路故障智能路由收敛方法[J]. 通信学报, 2022, 43(2): 131-142.
[13]	陈晋音, 胡书隆, 邢长友, 张国敏. 面向智能渗透攻击的欺骗防御方法[J]. 通信学报, 2022, 43(10): 106-120.
[14]	苏新, 孟蕾蕾, 周一青, CELIMUGE Wu. 基于深度强化学习的海洋移动边缘计算卸载方法[J]. 通信学报, 2022, 43(10): 133-145.
[15]	杜丽娜, 卓力, 杨硕, 李嘉锋, 张菁. 基于强化学习的移动视频流业务码率自适应算法研究进展[J]. 通信学报, 2021, 42(9): 205-217.