正则化流形信息极端学习机

doi:10.11959/j.issn.1000-436x.2016213

摘要/Abstract

摘要：

基于流形学习的思想和理论方法，提出刻画流形信息的正则化的极端学习机(MELM)算法。该算法利用流形信息刻画数据的几何结构和判别信息，克服 ELM 在有限样本上学习不充分的问题；能够有效提取数据样本的判别信息避免数据样本信息重叠；利用最大边际准则有效解决类间散度矩阵和类内散度矩阵的奇异问题。为验证所提方法的有效性，实验使用普遍应用的图像数据，将 MELM 与 ELM 以及相关最新算法 RAFELM、GELM进行识别率和计算效率的对比。实验结果表明，该算法能够显著提高 ELM 的分类准确率和泛化能力，并且优于其他相关算法。

关键词: 极端学习机, 几何结构, 流形信息, 机器学习

Abstract:

By exploiting the thought of manifold learning and its theoretical method, a regularized manifold information ex-treme learning machine algorithm aimed to depict and fully utilize manifold information was proposed. The proposed algo-rithm exploited the geometry and discrimination manifold information of data to perform network of ELM. The proposed algorithm could overcome the problem of the overlap of information. Singular problems of inter-class and within-class were solved effectively by using maximum margin criterion. The problem of inadequate learning with limited samples was solved. In order to demonstrate the effectiveness, comparative experiments with ELM and the related update algorithms RAFELM, GELM were conducted using the commonly used image data. Experimental results show that the proposed algorithm can significantly improve the generalization performance of ELM and outperforms the related update algorithms.

Key words: extreme learning machine, geometry, manifold information, machine learning

刘德山,楚永贺,闫德勤. 正则化流形信息极端学习机[J]. 通信学报, 2016, 37(11): 57-67.

De-shan LIU,Yong-he CHU,De-qin YAN. Regularized manifold information extreme learning machine[J]. Journal on Communications, 2016, 37(11): 57-67.

图/表 13

图1

表1

图2

图3

图4

图5

图6

表2

图7

表3

图8

图9

表4

参考文献 23

[1]	TANG J , DENG C , HUANG G B . Extreme learning machine for multilayer perception[J]. IEEE Transactions on Neural Networks and Learning Systems, 2015,(99):1-13.
[2]	HUANG G B , ZHOU H M , DING X J , et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2012,42(2):513-529.
[3]	HUANG G B , CHEN L , SIEW C K . Universal approximation using incremental constructive feedforward networks with random hidden nodes[J]. IEEE Trans Neural Networks, 2006,17(4):879-892.
[4]	ZHANG R , LAN Y , HUANG G B , et al. Universal approximation of extreme learning machine with adaptive growth of hidden nodes[J]. IEEE Trans Neural Networks Learn Systems, 2012,23(2):365-371.
[5]	FERNáNDEZ-DELGADO M , CERNADAS E , BARRO S , et al. Direct kernel perceptron (DKP): ultra-fast kernel ELM-based classifi-cation with noniterative closed-form weight calculation[J]. Neural Networks, 2014,50:60-71.
[6]	HUANG G , SONG S J . Semi-supervised and unsupervised extreme learning machines[J]. IEEE Transactions on Cybernetics, 2014,44(12):2405-2417.
[7]	HUANG G B , ZHU Q Y , SIEW C K . Extreme learning machine:theory and applications[J]. Neurocomputing. 2006,70:489-501.
[8]	VAPNIK V N . Statistical learning theory[J]. Encyclopedia of the Sciences of Learning, 2010,41(4):3185-3185.
[9]	HUANG G B , ZHOU H M , DING X J , et al. Extreme learning ma-chine for regression and multiclass classification[J]. IEEE Transac-tions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2012,42(2):513-529.
[10]	WANG X Z , SHAO Q Y , MIAO Q , et al. Architecture selection for networks trained with extreme learning machine using localized gen-eralization error model[J]. Neurocomputing, 2013,102:3-9.
[11]	ZHAO J W , WANG Z H , PARK D S . Online sequential extreme learning machine with forgetting mechanism[J]. Neurocomputing, 2012,87:79-89.
[12]	RONG H J , HUANG G B , SUNDARARAJAN N , et al. Online se-quential fuzzy extreme learning machine for function approximation and classification problems[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2009,39(4):1067-1072.
[13]	LIANG N Y , HUANG G B , SARATCHANDRAN P , et al. A fast and accurate online sequential learning algorithm for feedforward networks[J]. IEEE Transactions on Neural Networks, 2006,17(6):1411-1423.
[14]	ZONG W W , HUANG G B , CHEN Y . Weighted extreme learning machine for imbalance learning[J]. Neurocomputing, 2013,101:229-242.
[15]	YU Q , MICHE Y , EIROLA E , et al. Regularized extreme learning machine for regression with missing data[J]. Neurocomputing, 2013,102:45-51.
[16]	MAN Z H , WANG D H , CAO Z W , et al. Robust single-hidden layer feedforward network-based pattern classifier[J]. IEEE Transactions on Neural Networks and Learning Systems, 2012,23(12):1974-1986.
[17]	PENG Y , LU B L . Discriminative graph regularized extreme learning machine and its application to face recognition[J]. Neurocomputing, 2015,149:340-353.
[18]	BELKIN M , NIYOGI P . Laplacian eigenmaps for dimensionality reduction and data representation[J]. Neural Computation, 2003,15(6):1373-1396.
[19]	HE X , NIYOGI P . Locality preserving projections[J]. Advances in Neural Information Processing Systems, 2004,17:153-160.
[20]	LI H , JIANG T , ZHANG K . Efficient robust feature extraction by maximum margin criterion[J]. Advances in Neural Information Proc-essing Systems, 2003,144:71-78
[21]	LIU S , FENG L , XIAO Y . Robust activation function and its applica-tion: semi-supervised kernel extreme learning method[J]. Neurocom-puting, 2014,144:318-328.
[22]	HUANG G B . An insight into extreme learning machines: random neurons, random features and kernels[J]. Cognitive Computation, 2014,6:376-390.
[23]	HINTON G E , OSINDERO S . A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006,18(7):1527-1554.

数据集	维数	样本总数	每类样本数	类别数
Yale	1 024	165	11	15
Yale B	1 024	2 414	55	38
ORL	1 024	400	10	40
UMIST	10 305	575	18	20
MINST	784	70 000	7 000	10

数据集	ELM	RAFELM	GELM	MELM
Yale	0.025 3	0.234 0	0.670 8	0.276 9
ORL	0.062 4	0.456 3	1.721 8	0.522 6
Yale B	0.197 6	3.918 2	34.268 2	0.798 2
UMIST	0.052 0	0.512 2	1.814 8	0.270 4

数据集	DBN		CNN		MELM
数据集	最大识别率	平均识别率	最大识别率	平均识别率	最大识别率	平均识别率
Yale	80.00%	46.00%	70.00%	52.26%	86.67%	69.95%
ORL	51.50%	44.76%	25.00%	11.30%	95.00%	86.06%
UMIST	85.00%	57.27%	80.00%	26.82%	98.75%	83.40%
MINST	90.60%	81.32%	67.46%	35.17%	89.48%	85.78%

数据集	λ=0	λ=0.1	λ=0.2	λ=0.3	λ=0.4	λ=0.5	λ=0.6	λ=0.7	λ= 0.8	λ= 0.9
Yale	71.33%	60.00%	74.00%	74.67%	76.67%	70.67%	70.67%	69.33%	62.67%	64.67%
ORL	88.00%	87.50%	89.25%	91.25%	88.25%	94.75%	91.75%	90.75%	91.25%	90.50%
Yale B	93.21%	94.26%	94.74%	95.42%	93.53%	95.26%	95.11%	93.79%	94.47%	94.00%
UMIST	92.78%	89.17%	91.94%	92.22%	76.94%	89.17%	91.67%	91.94%	92.50%	90.83%