基于图正则化的贝叶斯宽度学习系统

doi:10.11959/j.issn.2096-6652.202203

Abstract

Abstract:

As a feed forward neural network, broad learning system (BLS) has attracted much attention because of its high accuracy, fast training speed, and the ability to effectively replace deep learning methods.However, it is sensitive to the number of feature nodes and the pseudo-inverse method is likely to result in the problem of over fitting for BLS model.To address the above issues, Bayesian inference and graph regularization was introduced in to the BLS model.By introducing the prior knowledge for Bayesian learning, the sparsity of the weights and the stability of the model could be effectively improved; while the graph information mining from the data could be fully considered to improve the generalization ability of the model by regularization.The UCI and NORB dataset were adopted for evaluating the performance of the proposed model.The experiment results demonstrated that the proposed graph-regularized Bayesian broad learning system model can further improve the accuracy of classification and has better stability.

Key words: board learning system, Bayesian inference, graph regularization, pattern recognition

CLC Number:

TP181

Junwei DUAN, Lincan XU, Yujuan QUAN, et al. Graph-regularized Bayesian broad learning system[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 109-117.

Figures/Tables 7

数据集	GBBLS参数				BBLS参数				BLS参数				BELM参数
数据集	准确率	Nf	Nm	Ne	准确率	Nf	Nm	Ne	准确率	Nf	Nm	Ne	准确率	Hn
Breast Cancer	$79 . 59 %$	8	1	6	78.91%	2	6	6	78.91%	2	5	7	74.15%	10
Climate Model	$94 . 44 %$	5	4	13	94.07%	5	6	8	12.22%	2	3	19	91.85%	15
Fertility	$91 . 22 %$	8	1	13	91.22%	9	3	1	36.84%	8	2	18	80.7%	5
Liver Disorders	$69 . 83 %$	6	10	4	68.71%	6	10	2	53.07%	1	4	7	65.92%	7
King-Rook	$96 . 03 %$	9	9	8	95.96%	10	5	8	94.95%	10	7	18	91.94%	20
Forest	$91 . 5 %$	5	3	6	91.5%	3	8	1	36.29%	1	1	4	90.35%	20
Glass	$92 . 79 %$	10	10	3	92.79%	10	10	3	86.49%	8	7	8	83.78%	12
Segmentation	$90 . 9 %$	8	2	7	90%	8	3	2	90%	3	5	15	80.91%	20
Seeds	$96 . 36 %$	10	8	1	96.36%	10	8	1	92.27%	1	7	1	94.55%	9
Wine	$100 %$	5	10	5	100%	5	10	5	100%	5	8	3	96.94%	12

References 32

[1]	张钹 . 人工智能进入后深度学习时代[J]. 智能科学与技术学报, 2019,1(1): 4-6.
	ZHANG B . Artificial intelligence is entering the post deep-learning era[J]. Chinese Journalof Intelligent Science and Technology, 2019,1(1): 4-6.
[2]	于璠 . 新一代深度学习框架研究[J]. 大数据, 2020,6(4): 69-80.
	YU F . Research on the next-generation deep learning framework[J]. Big Data Research, 2020,6(4): 69-80.
[3]	BROOMHEAD D S , LOWE D . Radial basis functions,multi-variable functional interpolation and adaptive networks[R]. 1988.
[4]	KOHONEN T . Self-organized formation of topologically correct feature maps[J]. Biological Cybernetics, 1982,43(1): 59-69.
[5]	ELMAN J L . Finding structure in time[J]. Cognitive Science, 1990,14(2): 179-211.
[6]	ACKLEY D H , HINTON G E , SEJNOWSKI T J . A learning algorithm for boltzmann machines[J]. Cognitive Science, 1985,9(1): 147-169.
[7]	HINTON G E , OSINDERO S , TEH Y W . A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006,18(7): 1527-1554.
[8]	YANN L , YOSHUA B . Convolutional networks for images,speech,and time-series[J]. The Handbook of Brain Theory and Neural Networks, 1995: 255-258.
[9]	黄宜华 . 大数据机器学习系统研究进展[J]. 大数据, 2015,1(1): 35-54.
	HUANG Y H . Research progress on big data machine learning sys-tem[J]. Big Data Research, 2015,1(1): 35-54.
[10]	郑南宁 . 人工智能新时代[J]. 智能科学与技术学报, 2019,1(1): 1-3.
	ZHENG N N . The new era of artificial intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 1-3.
[11]	孙长银, 穆朝絮 . 多智能体深度强化学习的若干关键科学问题[J]. 自动化学报, 2020,46(7): 1301-1312.
	SUN C Y , MU C X . Important scientific problems of multi-agent deep reinforcement learning[J]. Acta AutomaticaSinica, 2020,46(7): 1301-1312.
[12]	CHEN C L P , LIU Z L . Broad learning system:an effective and efficient incremental learning system without the need for deep architecture[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018,29(1): 10-24.
[13]	CHEN C L P , LIU Z L , FENG S . Universal approximation capability of broad learning system and its structural variations[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019,30(4): 1191-1204.
[14]	郑云飞, 陈霸东 . 基于最小p-范数的宽度学习系统[J]. 模式识别与人工智能, 2019,32(1): 51-57.
	ZHENG Y F , CHEN B D . Least p-norm based broad learning sys-tem[J]. Pattern Recognition and Artificial Intelligence, 2019,32(1): 51-57.
[15]	YANG F , . A CNN-based broad learning system[C]// Proceedings of 2018 IEEE 4th International Conference on Computer and Communications. Piscataway:IEEE Press, 2018: 2105-2109.
[16]	FENG S , CHEN C L P . Fuzzy broad learning system:a novel neuro-fuzzy model for regression and classification[J]. IEEE Transactions on Cybernetics, 2020,50(2): 414-424.
[17]	李润泽, 郭黎, 李保平 ,等. 基于模糊宽度学习模型的光伏发电预测方法[J]. 控制工程, 2020,27(11): 2016-2022.
	LI R Z , GUO L , LI B P ,et al. A photovoltaic power prediction method based on fuzzy broad learning system[J]. Control Engineering of Chi-na, 2020,27(11): 2016-2022.
[18]	XU L L , CHEN C L P , HAN R Z . Sparse Bayesian broad learning system for probabilistic estimation of prediction[J]. IEEE Access, 2020,8: 56267-56280.
[19]	JIN J W , LIU Z L , CHEN C L P . Discriminative graph regularized broad learning system for image recognition[J]. Science China Information Sciences, 2018,61(11): 112209.
[20]	贾晨, 刘华平, 续欣莹 ,等. 基于宽度学习方法的多模态信息融合[J]. 智能系统学报, 2019,14(1): 150-157.
	JIA C , LIU H P , XU X Y ,et al. Multi-modal information fusion based on broad learning method[J]. CAAI Transactions on Intelligent Sys-tems, 2019,14(1): 150-157.
[21]	褚菲, 苏嘉铭, 梁涛 ,等. 基于lasso和elastic net的宽度学习系统网络结构稀疏方法[J]. 控制理论与应用, 2020,37(12): 2543-2550.
	CHU F , SU J M , LIANG T ,et al. Sparsity method for network struc-ture of broad learning system based on lasso and elastic net[J]. Control Theory ＆ Applications, 2020,37(12): 2543-2550.
[22]	YANG L , SONG S J , CHEN C L P . Transductive transfer learning based on broad learning system[C]// Proceedings of 2018 IEEE International Conference on Systems,Man,and Cybernetics (SMC). Piscataway:IEEE Press, 2018: 912-917.
[23]	李旺, 俞祝良 . 宽度学习系统在蘑菇毒性判别中的应用[J]. 现代食品科技, 2019,35(7): 267-272,54.
	LI W , YU Z L . Application of broad learning system in discrimination of mushroom toxicity[J]. Modern Food Science and Technology, 2019,35(7): 267-272,54.
[24]	SORIA-OLIVAS E , GOMEZ-SANCHIS J , MARTIN J D ,et al. BELM:Bayesian extreme learning machine[J]. IEEE Transactions on Neural Networks, 2011,22(3): 505-509.
[25]	LUO J H , VONG C M , WONG P K . Sparse Bayesian extreme learning machine for multi-classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2014,25(4): 836-843.
[26]	BISHOP C . Pattern recognition and machine learning[M]. New York: Springer, 2006.
[27]	DIRKSE S P , FERRIS M C . A pathsearch damped Newton method for computing general equilibria[J]. Annals of Operations Research, 1996,68(2): 211-232.
[28]	BANERJEE K S . Generalized inverse of matrices and its applications[J]. Technometrics, 1973,15(1): 197.
[29]	KHAN M M R , ARIF R B , SIDDIQUE M Q B ,et al. Study and observation of the variation of the variation of accuracies of KNN,SVM,LMNN,ENN algorithms on eleven different datasets from UCI machine learning repository[C]// Proceedings of 2018 4th International Conference on Electrical Engineering and Information ＆ Communication Technology.[S.l.:s.n.], 2018: 124-129.
[30]	LECUN Y , HUANG F J , BOTTOU L . Learning methods for generic object recognition with invariance to pose and lighting[C]// Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2004.
[31]	DEBNATH R , TAKAHIDE N , TAKAHASHI H . A decision based one-against-one method for multi-class support vector machine[J]. Pattern Analysis and Applications, 2004,7(2): 164-175.
[32]	LI R , WANG X D , LEI L ,et al. Probabilities modeling of multi-class based on relevance vector machine[C]// Proceedings of 2016 12th World Congress on Intelligent Control and Automation. Piscataway:IEEE Press, 2016: 2755-2759.

Metrics

Recommended 0

No Suggested Reading articles found!

Graph-regularized Bayesian broad learning system

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 32

Related Articles 1

Metrics

Recommended 0

数据集	样本数/个	特征数/个	类别/个
Breast Cancer	683	10	2
Climate Model	540	18	2
Fertility	100	10	2
Liver Disorders	345	6	2
King-Rook	3 196	6	2
Forest	523	27	3
Glass	214	10	6
Segmentation	210	19	7
Seeds	210	7	3
Wine	178	13	3

模型	准确率
GBBLS	91.82%
BBLS	91.55%
BLS	92.01%
BELM	68.61%