支持合并的自适应tile coding算法

doi:10.11959/j.issn.1000-436x.2015047

Abstract

Abstract:

In order to solve many unnecessary division,mergence supported adaptive tile coding algorithm was presented which would eliminate the unnecessary division.Simulation is conducted on mountain car problem with discrete actions and continuous state space Results show that the proposed method can eliminate the influence of false division in the traditional tile coding method and achieve a more accurate adaptive partition of continuous state space.A higher convergence rate is achieved at the same time.

Key words: continuous space, discretization, reinforcement learning, adaptive; tile coding

Meng-yu SHI,Quan LIU,Qi-ming FU. Mergeable adaptive tile coding method[J]. Journal on Communications, 2015, 36(2): 186-192.

Figures/Tables 10

References 14

1	SUTTON R S , BARTO A G . Reinforcement Learning:An Introduction[M]. Cambridge:MIT Press, 1998.
2	LIN C S , KIM H . Selection of learning parameters for CMAC-based adaptive critic learning[J]. IEEE Trans Neural Networks, 1999,6(3):642-647.
3	PELLEG D , MOORE A , SHROFF N B . X-means:extending K-Means with efficient estimation of the number of clusters[A]. Proc of the 17th International Conf on Machine Learning[C]. Boston:Morgan Kaufmann Press, 2000.727-734.
4	PELLEG D , MOORE A . Accelerating exact k-means algorithms with geometric reasoning[A]. Proc of the fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining[C]. 1999.277-281.
5	陈宗海, 文锋, 聂建斌等. 基于节点生长 k-均值聚类算法的强化学习方法[J]. 计算机研究与发展, 2006,43(4):661-666. CHEN Z H , WEN F , NIE J B , et al. A reinforcement learning method based on node-growing k-means cluster algorithm[J]. Journal of Computer Research and Development, 2006,43(4):661-666.
6	文锋, 陈宗海, 卓睿等. 连续状态自适应离散化基于K-均值聚类的强化学习方法[J]. 控制与决策, 2006,21(2):143-147. WEN F , CHEN Z H , ZHUO R , et al. Reinforcement learning method of continuous state adaptively discretized based on K-means clustering[J]. Control and Decision, 2006,21(2):143-147.
7	顾冬雷, 陈卫东, 席裕庚 . 一种基于增强学习的自适应控制方法[J]. 控制与决策, 2002,17(4):473-479. GU D L , CHEN W D , XI Y G . A novel adaptive control algorithm based on reinforcement learning[J]. Control and Decision, 2002,17(4):473-479.
8	MOORE A W , ATKESON C G . The parti-game algorithm for variable resolution reinforcement learning in multidimensional state spaces[J]. Machine Learning, 1995,21(3):199-233.
9	UTHER W T B , VELOSO M M . Tree based discretization for continuous state space reinforcement learning[A]. AAAI’98[C]. Madison,Wisconsin,United States, 1998
10	SHERSTOV A A , STONE P . Function Approximation Via Tile Coding:Automating Parameter Choice Abstraction,Reformulation and Approximation[M]. Springer Berlin Heidelberg, 2005:194-205.
11	WHITESON S , TAYLOR M E , STONE P . Adaptive tile Coding for Value Function Approximation[M]. Computer Science Department,University of Texas at Austin, 2007.
12	WHITESON S , STONE P . Evolutionary function approximation for reinforcement learning[J]. The Journal of Machine Learning Research, 2006,7:877-917.
13	NOKHBEH-ZAEEM M , KHASHABI D , TALEBI H A , et al. Adaptive tiled neural networks[A]. 2011 IEEE International Conference on Systems,Man,and Cybernetics (SMC)[C]. New Orleans,LA,USA, 2011.2543-2548.
14	LIN S , WRIGHT R . Evolutionary tile coding:an automated state abstraction algorithm for reinforcement learning[A]. AAAI Workshops[C]. 2010.

Metrics

Recommended 0

No Suggested Reading articles found!

tile	ATC情节数	MATC情节数
60	593	168
100	735	247
200	2 200	892
300	6 492	1 217

[1]	Ling MA, Qiliang FAN, Ting XU, Guanchen GUO, Shenglin ZHANG, Yongqian SUN, Yuzhi ZHANG. Scheduling framework based on reinforcement learning in online-offline colocated cloud environment [J]. Journal on Communications, 2023, 44(6): 90-102.
[2]	Biao JIN, Yikang LI, Zhiqiang YAO, Yulin CHEN, Jinbo XIONG. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents [J]. Journal on Communications, 2023, 44(6): 183-197.
[3]	Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform [J]. Journal on Communications, 2023, 44(5): 181-192.
[4]	Dacheng ZHOU, Hongchang CHEN, Weizhen HE, Guozhen CHENG, Hongchao HU. Research on multidimensional dynamic defense strategy for microservice based on deep reinforcement learning [J]. Journal on Communications, 2023, 44(4): 50-63.
[5]	Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system [J]. Journal on Communications, 2023, 44(4): 78-86.
[6]	Wenjun XU, Silei WU, Fengyu WANG, Lan LIN, Guojun LI, Zhi ZHANG. Large-scale post-disaster user distributed coverage optimization based on multi-agent reinforcement learning [J]. Journal on Communications, 2022, 43(8): 1-16.
[7]	Zongxuan SHA, Ru HUO, Chuang SUN, Shuo WANG, Tao HUANG. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 30-40.
[8]	Shuai MA, Bing LI, Haihong SHENG, Rongyan GU, Hui ZHOU, Hongmei WANG, Yue WANG, Shiyin LI. Research on power allocation of integrated VLPC based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 121-130.
[9]	Yu ZHANG, Min CHENG. Joint optimization of edge computing and caching in NDN [J]. Journal on Communications, 2022, 43(8): 164-175.
[10]	Peiliang ZUO, Shaolong HOU, Chao GUO, Hua JIANG, Wenbo WANG. Security decision method for the edge of multi-layer satellite network based on reinforcement learning [J]. Journal on Communications, 2022, 43(6): 189-199.
[11]	Xianchao ZHANG, Yao ZHAO, Haijun YE, Rui FAN. Intelligent transmit power control algorithm for the multi-user interference of wireless network [J]. Journal on Communications, 2022, 43(2): 15-21.
[12]	Chuanhuang LI, Yangting CHEN, Jingjing TANG, Jiali LOU, Renhua XIE, Chuntao FANG, Weiming WANG, Chao CHEN. QL-STCT: an intelligent routing convergence method for SDN link failure [J]. Journal on Communications, 2022, 43(2): 131-142.
[13]	Jinyin CHEN, Shulong HU, Changyou XING, Guomin ZHANG. Deception defense method against intelligent penetration attack [J]. Journal on Communications, 2022, 43(10): 106-120.
[14]	Xin SU, Leilei MENG, Yiqing ZHOU, Wu CELIMUGE. Maritime mobile edge computing offloading method based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(10): 133-145.
[15]	Li’na DU, Li ZHUO, Shuo YANG, Jiafeng LI, Jing ZHANG. Survey on reinforcement learning based adaptive bit rate algorithm for mobile video streaming services [J]. Journal on Communications, 2021, 42(9): 205-217.

Mergeable adaptive tile coding method

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 14

Related Articles 15

Metrics

Recommended 0