一类非仿射系统的执行依赖启发式在线跟踪控制

doi:10.11959/j.issn.2096-6652.202144

智能科学与技术学报 ›› 2021, Vol. 3 ›› Issue (4): 449-455.doi: 10.11959/j.issn.2096-6652.202144

• 专栏：基于数据的学习与优化 • 上一篇下一篇

一类非仿射系统的执行依赖启发式在线跟踪控制

赵慧玲¹^,²^,³^,⁴, 王鼎¹^,²^,³^,⁴, 任进¹^,²^,³^,⁴

¹ 北京工业大学信息学部，北京 100124
² 计算智能与智能系统北京市重点实验室，北京 100124
³ 北京人工智能研究院，北京 100124
⁴ 智慧环保北京实验室，北京 100124

修回日期:2021-11-20 出版日期:2021-12-15 发布日期:2021-12-01
作者简介:赵慧玲（1998– ），女，北京工业大学信息学部硕士生，主要研究方向为强化学习与智能控制
王鼎（1984– ），男，博士，北京工业大学信息学部教授、博士生导师，主要研究方向为强化学习与智能控制
任进（1999– ），女，北京工业大学信息学部硕士生，主要研究方向为强化学习与智能控制
基金资助:
国家自然科学基金资助项目(61773373);国家自然科学基金资助项目(61890930-5);国家自然科学基金资助项目(62021003);国家重点研发计划资助项目(2021ZD0112300-2);国家重点研发计划资助项目(2018YFC1900800-5);北京市自然科学基金资助项目(JQ19013)

Action dependent heuristic online tracking control for a class of nonaffine systems

Huiling ZHAO¹^,²^,³^,⁴, Ding WANG¹^,²^,³^,⁴, Jin REN¹^,²^,³^,⁴

¹ Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
² Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing 100124, China
³ Beijing Institute of Artificial Intelligence, Beijing 100124, China
⁴ Beijing Laboratory of Smart Environmental Protection, Beijing 100124, China

Revised:2021-11-20 Online:2021-12-15 Published:2021-12-01
Supported by:
The National Natural Science Foundation of China(61773373);The National Natural Science Foundation of China(61890930-5);The National Natural Science Foundation of China(62021003);The National Key Research and Development Project of China(2021ZD0112300-2);The National Key Research and Development Project of China(2018YFC1900800-5);Beijing Natural Science Foundation(JQ19013)

摘要/Abstract

摘要：

针对非仿射系统的跟踪控制问题，提出一种基于执行依赖启发式动态规划（ADHDP）结构的在线设计方法。首先，考虑一类未知非仿射系统，进行从跟踪控制问题到误差调节问题的转换；然后，设计基于 ADHDP结构的跟踪控制器，并采用在线学习的方式，实现系统控制与执行网络和评判网络的训练同步，使得系统状态能够跟踪期望轨迹；最后，通过一个仿真实例验证所提方法的有效性。

关键词: 跟踪控制, 在线学习, 执行依赖设计

Abstract:

To solve the tracking control problem for nonaffine systems, an online design method was developed by using the action dependent heuristic dynamic programming (ADHDP) structure.Firstly, the tracking control problem for the unknown nonaffine system was transformed into the error regulation problem.Then, the ADHDP tracking controller was designed and the online learning method was adopted to synchronize the system control with the training of action networks and critic networks, so that the desired trajectory could be tracked by the system state.Finally, a simulation example was given to verify the effectiveness of the proposed method.

Key words: tracking control, online learning, action dependent design

中图分类号:

TP13

赵慧玲,王鼎,任进. 一类非仿射系统的执行依赖启发式在线跟踪控制[J]. 智能科学与技术学报, 2021, 3(4): 449-455.

Huiling ZHAO,Ding WANG,Jin REN. Action dependent heuristic online tracking control for a class of nonaffine systems[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 449-455.

图/表 7

图1

图2

表1

基于ADHDP结构的在线控制方法的参数值"

$Q$	R	λ _c、λ_a	N _c、N_a	ρ _c、ρ_a
$[\begin{matrix} 0.01 & 0 \\ 0 & 0.5 \end{matrix}]$	0.01	0.05	1 000	1×10^{- 12}

表1

图3

图4

图5

图6

参考文献 17

[1]	LIU D R , WEI Q L . Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems[J]. IEEE Transactions on Neural Networks and Learning Systems, 2014,25(3): 621-634.
[2]	LIU D R , WEI Q L , WANG D ,et al. Adaptive dynamic programming for optimal residential energy management[M]. Adaptive dynamic programming with applications in optimal control. Cham: Springer International Publishing, 2017: 483-535.
[3]	WANG D , HA M M , QIAO J F . Self-learning optimal regulation for discrete-time nonlinear systems under event-driven formulation[J]. IEEE Transactions on Automatic Control, 2020,65(3): 1272-1279.
[4]	WANG D , QIAO J F , CHENG L . An approximate neuro-optimal solution of discounted guaranteed cost control design[J]. IEEE Transactions on Cybernetics, 2020: 1-10.
[5]	LI Y M , LIU Y J , TONG S C . Observer-based neuro-adaptive optimized control of strict-feedback nonlinear systems with state constraints[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021: 1-15.
[6]	YANG X , HE H B , ZHONG X N . Approximate dynamic programming for nonlinear-constrained optimizations[J]. IEEE Transactions on Cybernetics, 2021,51(5): 2419-2432.
[7]	WANG D , HA M M , CHENG L . Neuro-optimal trajectory tracking with value iteration of discrete-time nonlinear dynamics[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021: 1-12.
[8]	WANG D , LIU D R , WEI Q L . Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach[J]. Neurocomputing, 2012,78(1): 14-22.
[9]	KIUMARSI B , LEWIS F L . Actor-critic-based optimal tracking for partially unknown nonlinear discrete-time systems[J]. IEEE Transactions on Neural Networks and Learning Systems, 2015,26(1): 140-151.
[10]	HA M M , WANG D , LIU D R . Data-based nonaffine optimal tracking control using iterative DHP approach[J]. IFAC-PapersOnLine, 2020,53(2): 4246-4251.
[11]	WANG D , HA M M , QIAO J F . Data-driven iterative adaptive critic control toward an urban wastewater treatment plant[J]. IEEE Transactions on Industrial Electronics, 2021,68(8): 7362-7369.
[12]	WEI Q L , LIAO Z H , SONG R Z ,et al. Self-learning optimal control for ice-storage air conditioning systems via data-based adaptive dynamic programming[J]. IEEE Transactions on Industrial Electronics, 2021,68(4): 3599-3608.
[13]	李金娜, 程薇燃 . 基于强化学习的数据驱动多智能体系统最优一致性综述[J]. 智能科学与技术学报, 2020,2(4): 327-340.
	LI J N , CHENG W R . An overview of optimal consensus for data driven multi-agent system based on reinforcement learning[J]. Chinese Journal of Intelligent Science and Technology, 2020,2(4): 327-340.
[14]	李涛, 魏庆来 . 基于深度强化学习的智能暖气温度控制系统[J]. 智能科学与技术学报, 2020,2(4): 348-353.
	LI T , WEI Q L . Intelligent heating temperature control system based on deep reinforcement learning[J]. Chinese Journal of Intelligent Science and Technology, 2020,2(4): 348-353.
[15]	王金予, 魏欣然, 石文磊 ,等. 强化学习在资源优化领域的应用[J]. 大数据, 2021,7(5): 131-149.
	WANG J Y , WEI X R , SHI W L ,et al. Applications of reinforcement learning in the field of resource optimization[J]. Big Data Research, 2021,7(5): 131-149.
[16]	LUO B , LIU D R , HUANG T W ,et al. Model-free optimal tracking control via critic-only Q-learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016,27(10): 2134-2144.
[17]	LI S , DING L , GAO H B ,et al. ADP-based online tracking control of partially uncertain time-delayed nonlinear system and application to wheeled mobile robots[J]. IEEE Transactions on Cybernetics, 2020,50(7): 3182-3194.

一类非仿射系统的执行依赖启发式在线跟踪控制

Action dependent heuristic online tracking control for a class of nonaffine systems

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 17

相关文章 3

Metrics

推荐阅读 0

[1]	张向文, 王飞跃. 平行轮胎的基本架构与关键技术[J]. 智能科学与技术学报, 2022, 4(3): 445-457.
[2]	王飞跃. 平行控制与数字孪生：经典控制理论的回顾与重铸[J]. 智能科学与技术学报, 2020, 2(3): 293-300.
[3]	侯家琛,董西松,熊刚,张俊,谭珂. 平行核电：迈向智慧核电的智能技术[J]. 智能科学与技术学报, 2019, 1(2): 192-201.