基于图正则化的贝叶斯宽度学习系统

doi:10.11959/j.issn.2096-6652.202203

智能科学与技术学报 ›› 2022, Vol. 4 ›› Issue (1): 109-117.doi: 10.11959/j.issn.2096-6652.202203

基于图正则化的贝叶斯宽度学习系统

段俊伟¹, 许林灿¹, 全渝娟¹, 陈龙², 陈俊龙³

¹ 暨南大学信息科学技术学院，广东广州 510632
² 澳门大学科技学院，澳门 999078
³ 华南理工大学计算机科学与工程学院，广东广州 510006

修回日期:2021-04-17 出版日期:2022-03-15 发布日期:2022-03-01
作者简介:段俊伟（1984− ），男，博士，暨南大学信息科学技术学院讲师，主要研究方向为人工智能与大数据、信息融合、图像处理、智能系统及应用
许林灿（1999− ），男，暨南大学信息科学技术学院在读，主要研究方向为模式识别、深度学习
全渝娟（1968− ），女，博士，暨南大学信息科学技术学院副教授，主要研究方向为人工智能与推荐系统、物联网
陈龙（1978− ），男，博士，澳门大学科技学院副教授，主要研究方向为计算智能、机器学习方法及应用
陈俊龙（1959− ），男，博士，欧洲科学院院士，IEEE Fellow，华南理工大学计算机科学与工程学院院长，主要研究方向为智能系统与控制、计算智能、混合智能、无人系统、数据科学
基金资助:
国家重点研发计划基金资助项目(2018YFC2002500);广东省基础与应用基础研究基金资助项目(2021A1515011999);广州市科技创新发展专项资金项目(201902010041)

Graph-regularized Bayesian broad learning system

Junwei DUAN¹, Lincan XU¹, Yujuan QUAN¹, Long CHEN², C.L.Philip CHEN³

¹ College of Information Science and Technology, Jinan University, Guangzhou 510632, China
² Faculty of Science and Technology, University of Macau, Macau 999078, China
³ School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China

Revised:2021-04-17 Online:2022-03-15 Published:2022-03-01
Supported by:
The National Key Research and Development Program of China(2018YFC2002500);The Guangdong Basic and Applied Basic Research Foundation(2021A1515011999);The Guangzhou Science and Technology Innovation and Development Special Fund Project(201902010041)

摘要/Abstract

摘要：

作为一种前馈神经网络，宽度学习系统因其精度高、训练速度快且能有效代替深度学习方法而备受研究者的关注。然而，宽度学习系统存在对网络中的特征节点个数比较敏感且求伪逆方式易使模型出现过拟合等问题。为此，在宽度学习系统中引入贝叶斯推断和图正则化。一方面，通过引入先验知识进行贝叶斯学习可以有效提高权重的稀疏性，提高模型的稳定性；另一方面，加入图正则化可充分考虑数据内在的图信息，进一步提高模型的泛化能力。在UCI数据集和NORB数据集上对所提模型进行性能评估，实验结果表明，所提的基于图正则化的贝叶斯宽度学习系统模型能进一步提高宽度学习系统的分类精度且具有更好的稳定性。

关键词: 宽度学习系统, 贝叶斯推断, 图正则化, 模式识别

Abstract:

As a feed forward neural network, broad learning system (BLS) has attracted much attention because of its high accuracy, fast training speed, and the ability to effectively replace deep learning methods.However, it is sensitive to the number of feature nodes and the pseudo-inverse method is likely to result in the problem of over fitting for BLS model.To address the above issues, Bayesian inference and graph regularization was introduced in to the BLS model.By introducing the prior knowledge for Bayesian learning, the sparsity of the weights and the stability of the model could be effectively improved; while the graph information mining from the data could be fully considered to improve the generalization ability of the model by regularization.The UCI and NORB dataset were adopted for evaluating the performance of the proposed model.The experiment results demonstrated that the proposed graph-regularized Bayesian broad learning system model can further improve the accuracy of classification and has better stability.

Key words: board learning system, Bayesian inference, graph regularization, pattern recognition

中图分类号:

TP181

段俊伟, 许林灿, 全渝娟, 等. 基于图正则化的贝叶斯宽度学习系统[J]. 智能科学与技术学报, 2022, 4(1): 109-117.

Junwei DUAN, Lincan XU, Yujuan QUAN, et al. Graph-regularized Bayesian broad learning system[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 109-117.

图/表 7

图1

图2

表1

图3

表2

各模型在UCI中的10个数据集上的准确率"

数据集	GBBLS参数				BBLS参数				BLS参数				BELM参数
数据集	准确率	Nf	Nm	Ne	准确率	Nf	Nm	Ne	准确率	Nf	Nm	Ne	准确率	Hn
Breast Cancer	$79 . 59 %$	8	1	6	78.91%	2	6	6	78.91%	2	5	7	74.15%	10
Climate Model	$94 . 44 %$	5	4	13	94.07%	5	6	8	12.22%	2	3	19	91.85%	15
Fertility	$91 . 22 %$	8	1	13	91.22%	9	3	1	36.84%	8	2	18	80.7%	5
Liver Disorders	$69 . 83 %$	6	10	4	68.71%	6	10	2	53.07%	1	4	7	65.92%	7
King-Rook	$96 . 03 %$	9	9	8	95.96%	10	5	8	94.95%	10	7	18	91.94%	20
Forest	$91 . 5 %$	5	3	6	91.5%	3	8	1	36.29%	1	1	4	90.35%	20
Glass	$92 . 79 %$	10	10	3	92.79%	10	10	3	86.49%	8	7	8	83.78%	12
Segmentation	$90 . 9 %$	8	2	7	90%	8	3	2	90%	3	5	15	80.91%	20
Seeds	$96 . 36 %$	10	8	1	96.36%	10	8	1	92.27%	1	7	1	94.55%	9
Wine	$100 %$	5	10	5	100%	5	10	5	100%	5	8	3	96.94%	12

表2

表3

图4

参考文献 32

[1]	张钹 . 人工智能进入后深度学习时代[J]. 智能科学与技术学报, 2019,1(1): 4-6.
	ZHANG B . Artificial intelligence is entering the post deep-learning era[J]. Chinese Journalof Intelligent Science and Technology, 2019,1(1): 4-6.
[2]	于璠 . 新一代深度学习框架研究[J]. 大数据, 2020,6(4): 69-80.
	YU F . Research on the next-generation deep learning framework[J]. Big Data Research, 2020,6(4): 69-80.
[3]	BROOMHEAD D S , LOWE D . Radial basis functions,multi-variable functional interpolation and adaptive networks[R]. 1988.
[4]	KOHONEN T . Self-organized formation of topologically correct feature maps[J]. Biological Cybernetics, 1982,43(1): 59-69.
[5]	ELMAN J L . Finding structure in time[J]. Cognitive Science, 1990,14(2): 179-211.
[6]	ACKLEY D H , HINTON G E , SEJNOWSKI T J . A learning algorithm for boltzmann machines[J]. Cognitive Science, 1985,9(1): 147-169.
[7]	HINTON G E , OSINDERO S , TEH Y W . A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006,18(7): 1527-1554.
[8]	YANN L , YOSHUA B . Convolutional networks for images,speech,and time-series[J]. The Handbook of Brain Theory and Neural Networks, 1995: 255-258.
[9]	黄宜华 . 大数据机器学习系统研究进展[J]. 大数据, 2015,1(1): 35-54.
	HUANG Y H . Research progress on big data machine learning sys-tem[J]. Big Data Research, 2015,1(1): 35-54.
[10]	郑南宁 . 人工智能新时代[J]. 智能科学与技术学报, 2019,1(1): 1-3.
	ZHENG N N . The new era of artificial intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 1-3.
[11]	孙长银, 穆朝絮 . 多智能体深度强化学习的若干关键科学问题[J]. 自动化学报, 2020,46(7): 1301-1312.
	SUN C Y , MU C X . Important scientific problems of multi-agent deep reinforcement learning[J]. Acta AutomaticaSinica, 2020,46(7): 1301-1312.
[12]	CHEN C L P , LIU Z L . Broad learning system:an effective and efficient incremental learning system without the need for deep architecture[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018,29(1): 10-24.
[13]	CHEN C L P , LIU Z L , FENG S . Universal approximation capability of broad learning system and its structural variations[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019,30(4): 1191-1204.
[14]	郑云飞, 陈霸东 . 基于最小p-范数的宽度学习系统[J]. 模式识别与人工智能, 2019,32(1): 51-57.
	ZHENG Y F , CHEN B D . Least p-norm based broad learning sys-tem[J]. Pattern Recognition and Artificial Intelligence, 2019,32(1): 51-57.
[15]	YANG F , . A CNN-based broad learning system[C]// Proceedings of 2018 IEEE 4th International Conference on Computer and Communications. Piscataway:IEEE Press, 2018: 2105-2109.
[16]	FENG S , CHEN C L P . Fuzzy broad learning system:a novel neuro-fuzzy model for regression and classification[J]. IEEE Transactions on Cybernetics, 2020,50(2): 414-424.
[17]	李润泽, 郭黎, 李保平 ,等. 基于模糊宽度学习模型的光伏发电预测方法[J]. 控制工程, 2020,27(11): 2016-2022.
	LI R Z , GUO L , LI B P ,et al. A photovoltaic power prediction method based on fuzzy broad learning system[J]. Control Engineering of Chi-na, 2020,27(11): 2016-2022.
[18]	XU L L , CHEN C L P , HAN R Z . Sparse Bayesian broad learning system for probabilistic estimation of prediction[J]. IEEE Access, 2020,8: 56267-56280.
[19]	JIN J W , LIU Z L , CHEN C L P . Discriminative graph regularized broad learning system for image recognition[J]. Science China Information Sciences, 2018,61(11): 112209.
[20]	贾晨, 刘华平, 续欣莹 ,等. 基于宽度学习方法的多模态信息融合[J]. 智能系统学报, 2019,14(1): 150-157.
	JIA C , LIU H P , XU X Y ,et al. Multi-modal information fusion based on broad learning method[J]. CAAI Transactions on Intelligent Sys-tems, 2019,14(1): 150-157.
[21]	褚菲, 苏嘉铭, 梁涛 ,等. 基于lasso和elastic net的宽度学习系统网络结构稀疏方法[J]. 控制理论与应用, 2020,37(12): 2543-2550.
	CHU F , SU J M , LIANG T ,et al. Sparsity method for network struc-ture of broad learning system based on lasso and elastic net[J]. Control Theory ＆ Applications, 2020,37(12): 2543-2550.
[22]	YANG L , SONG S J , CHEN C L P . Transductive transfer learning based on broad learning system[C]// Proceedings of 2018 IEEE International Conference on Systems,Man,and Cybernetics (SMC). Piscataway:IEEE Press, 2018: 912-917.
[23]	李旺, 俞祝良 . 宽度学习系统在蘑菇毒性判别中的应用[J]. 现代食品科技, 2019,35(7): 267-272,54.
	LI W , YU Z L . Application of broad learning system in discrimination of mushroom toxicity[J]. Modern Food Science and Technology, 2019,35(7): 267-272,54.
[24]	SORIA-OLIVAS E , GOMEZ-SANCHIS J , MARTIN J D ,et al. BELM:Bayesian extreme learning machine[J]. IEEE Transactions on Neural Networks, 2011,22(3): 505-509.
[25]	LUO J H , VONG C M , WONG P K . Sparse Bayesian extreme learning machine for multi-classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2014,25(4): 836-843.
[26]	BISHOP C . Pattern recognition and machine learning[M]. New York: Springer, 2006.
[27]	DIRKSE S P , FERRIS M C . A pathsearch damped Newton method for computing general equilibria[J]. Annals of Operations Research, 1996,68(2): 211-232.
[28]	BANERJEE K S . Generalized inverse of matrices and its applications[J]. Technometrics, 1973,15(1): 197.
[29]	KHAN M M R , ARIF R B , SIDDIQUE M Q B ,et al. Study and observation of the variation of the variation of accuracies of KNN,SVM,LMNN,ENN algorithms on eleven different datasets from UCI machine learning repository[C]// Proceedings of 2018 4th International Conference on Electrical Engineering and Information ＆ Communication Technology.[S.l.:s.n.], 2018: 124-129.
[30]	LECUN Y , HUANG F J , BOTTOU L . Learning methods for generic object recognition with invariance to pose and lighting[C]// Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2004.
[31]	DEBNATH R , TAKAHIDE N , TAKAHASHI H . A decision based one-against-one method for multi-class support vector machine[J]. Pattern Analysis and Applications, 2004,7(2): 164-175.
[32]	LI R , WANG X D , LEI L ,et al. Probabilities modeling of multi-class based on relevance vector machine[C]// Proceedings of 2016 12th World Congress on Intelligent Control and Automation. Piscataway:IEEE Press, 2016: 2755-2759.

基于图正则化的贝叶斯宽度学习系统

Graph-regularized Bayesian broad learning system

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 32

相关文章 1

Metrics

推荐阅读 0

数据集	样本数/个	特征数/个	类别/个
Breast Cancer	683	10	2
Climate Model	540	18	2
Fertility	100	10	2
Liver Disorders	345	6	2
King-Rook	3 196	6	2
Forest	523	27	3
Glass	214	10	6
Segmentation	210	19	7
Seeds	210	7	3
Wine	178	13	3

模型	准确率
GBBLS	91.82%
BBLS	91.55%
BLS	92.01%
BELM	68.61%