稀疏诱导流形正则化凸非负矩阵分解算法

doi:10.11959/j.issn.1000-436x.2020064

Abstract

Abstract:

To address problems that the effectiveness of feature learned from real noisy data by classical nonnegative matrix factorization method,a novel sparsity induced manifold regularized convex nonnegative matrix factorization algorithm (SGCNMF) was proposed.Based on manifold regularization,the L<sub>2,1</sub>norm was introduced to the basis matrix of low dimensional subspace as sparse constraint.The multiplicative update rules were given and the convergence of the algorithm was analyzed.Clustering experiment was designed to verify the effectiveness of learned features within various of noisy environments.The empirical study based on K-means clustering shows that the sparse constraint reduces the representation of noisy features and the new method is better than the 8 similar algorithms with stronger robustness to a variable extent.

Key words: nonnegative matrix factorization, manifold regularization, sparse constraint, K-means clustering

CLC Number:

TP181

Feiyue QIU,Bowen CHEN,Tieming CHEN,Guodao ZHANG. Sparsity induced convex nonnegative matrix factorization algorithm with manifold regularization[J]. Journal on Communications, 2020, 41(5): 84-95.

Figures/Tables 10

References 35

[1]	DUDA R O , HART P E , STORK D G . Pattern classification,seconded[M]. New York: John Wiley ＆ SonsPress, 2001.
[2]	LEE D D , SEUNG H S . Learning the parts of objects by non-negative matrix factorization[J]. Nature, 1999,401(6755): 788-791.
[3]	WU W H , KWONG S , ZHOU Y ,et al. Nonnegative matrix factorization with mixed hypergraph regularization for community detection[J]. Information Sciences, 2018,435(4): 263-281.
[4]	CHEN G F , XU C , WANG J Y ,et al. Graph regularization weighted nonnegative matrix factorization for link prediction in weighted complex network[J]. Neurocomputing, 2019,369(12): 50-60.
[5]	常振超, 陈鸿昶, 黄瑞阳 ,等. 基于非负矩阵分解的半监督动态社团检测[J]. 通信学报, 2016,37(2): 132-143.
	CHANG Z C , CHEN H C , HUANG R Y ,et al. Semi-supervised dynamic community detection based on non-negative matrix factorization[J]. Journal on Communications, 2016,37(2): 132-143.
[6]	LIU R , DU B , ZHANG L P . Hyperspectral unmixing via double abundance characteristics constraints based NMF[J]. Remote Sensing, 2016,8(6): 1-23.
[7]	WANG H , YANG W J , GUAN N Y . Cauchy sparse NMF with manifold regularization:a robust method for hyperspectral unmixing[J]. Knowledge-Based Systems, 2019,184(104898): 1-16.
[8]	CHEN W S , LIU J , PAN B ,et al. Face recognition using nonnegative matrix factorization with fractional power inner product kernel[J]. Neurocomputing, 2019,348(7): 40-53.
[9]	YANG Z , CHEN W T , HUANG J . Enhancing recommendation on extremely sparse data with blocks-coupled non-negative matrix factorization[J]. Neurocomputing, 2018,278(2): 126-133.
[10]	LI S Z , HOU X W , ZHANG H J ,et al. Learning spatially localized,parts-based representation[C]// Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR).Piscataway:IEEE Press. 2001: 1-6.
[11]	ZHAN S , WU J , HAN N ,et al. Unsupervised feature extraction by low-rank and sparsity preserving embedding[J]. Neural Networks, 2019,109(1): 56-66.
[12]	CAI D , HE X F , HAN J H ,et al. Graph regularized nonnegative matrix factorization for data representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011,33(8): 1548-1560.
[13]	HUANG S D , WANG H J , LI T ,et al. Robust graph regularized nonnegative matrix factorization for clustering[J]. Data Mining and Knowledge Discovery, 2018,32(2): 483-503.
[14]	LI X L , CUI G , DONG Y . Graph regularized non-negative low-rank matrix factorization for image clustering[J]. IEEE Transactions on Cybernetics, 2016,47(11): 3840-3853.
[15]	WU B L , WANG E Y , ZHU Z ,et al. Manifold NMF with L-21 norm for clustering[J]. Neurocomputing, 2018,273(1): 78-88.
[16]	ZENG K , YU J , LI C ,et al. Image clustering by hyper-graph regularized non-negative matrix factorization[J]. Neurocomputing, 2014,138(8): 209-217.
[17]	WANG L , ZHANG Z , LIU G C ,et al. Robust adaptive low-rank and sparse embedding for feature representation[C]// International Conference on Pattern Recognition (ICPR). 2018: 800-805.
[18]	YI Y G , WANG J Z , ZHOU W ,et al. Non-negative matrix factorization with locality constrained adaptive graph[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020,30(2): 427-441.
[19]	LU G F , WANG Y , ZOU J . Low-rank matrix factorization with adaptive graph regularizer[J]. IEEE Transactions on Image Processing, 2016,25(5): 2196-2205.
[20]	YIN M , XIE S L , WU Z Z ,et al. Subspace clustering via learning an adaptive low-rank graph[J]. IEEE Transactions on Image Processing, 2018,27(8): 3716-3728.
[21]	DING C H Q , LI T , JORDAN M I . Convex and semi-nonnegative matrix factorizations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009,32(1): 45-55.
[22]	WANG S P , PEDRYCZ W , ZHU Q ,et al. Subspace learning for unsupervised feature selection via matrix factorization[J]. Pattern Recognition, 2015,48(1): 10-19.
[23]	HU W J , CHOI K S , WANG P L ,et al. Convex non-negative matrix factorization with manifold regularization[J]. Neural Networks, 2015,63(3): 94-103.
[24]	CUI G S , LI X L , DONG Y S . Subspace clustering guided convex nonnegative matrix factorization[J]. Neurocomputing, 2018,292(5): 38-48.
[25]	LI G P , ZHANG X Y , ZHENG S Y ,et al. Semi-supervised convex non-negative matrix factorizations with graph regularized for image representation[J]. Neurocomputing, 2017,237(5): 1-11.
[26]	LI Z C , LIU J , TANG J H ,et al. Robust structured subspace learning for data representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,37(10): 2085-2098.
[27]	KONG D G , DING C , HUANG H . Robust non-negative matrix factorization using L21-norm[C]// 20th ACM Conference on Information and Knowledge Management (CIKM). New York:ACM Press, 2011: 673-682.
[28]	HOYER P O . Non-negative matrix factorization with sparseness constraints[J]. Journal of Machine Learning Research, 2004,5(1): 1457-1469.
[29]	张旭, 陈志奎, 高静 . 基于图正则化和 l_(1/2)稀疏约束的非负矩阵分解算法[J]. 小型微型计算机系统, 2018,39(11): 130-134.
	ZHANG X , CHEN Z K , GAO J . Nonnegative matrix factorization via graph regularization and l 1 / 2 sparse constraints[J]. Journal of Chinese Computer Systems, 2018,39(11): 130-134.
[30]	LEE D D , SEUNG H S . Algorithms for non-negative matrix factorization[C]// Advances in Neural Information Processing Systems (NIPS). 2001: 556-562.
[31]	WANG Y X , ZHANG Y J . Nonnegative matrix factorization:a comprehensive review[J]. IEEE Transactions on Knowledge and Data Engineering, 2012,25(6): 1336-1353.
[32]	LUXBURG U V . A tutorial on spectral clustering[J]. Statistics and Computing, 2007,17(4): 395-416.
[33]	NIE F P , HUANG H , CAI X ,et al. Efficient and robust feature selection via joint ?2,1-norms minimization[C]// Advances in Neural Information Processing Systems (NIPS). 2010: 1813-1821.
[34]	HULL J J . A database for handwritten text recognition research[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994,16(5): 550-554.
[35]	CAI D , HE X F , HAN J H . Document clustering using locality preserving indexing[J]. IEEE Transactions on Knowledge and Data Engineering, 2005,17(12): 1624-1637.

Metrics

Recommended 0

No Suggested Reading articles found!

KCNMF	SGCNMF(α=0)	MFFS	HNMF	GNLMF	NMFLCAG SCCNMF	GCNMF	SGCNMF
448.12±7.53	50.75±8.13	48.22±7.53	89.88±12.56	85.38±12.78	69.56±10.8175.63±9.72	86.37±10.47	89.12±8.31
643.71±8.07	44.25±6.24	43.94±4.81	80.58±11.99	80.75±11.68	55.58±8.7168.58±6.95	79.79±11.89	83.15±10.51
840.94±7.84	39.06±6.46	41.04±5.78	74.44±9.15	74.13±11.87	51.73±2.1063.25±6.40	73.44±13.91	76.35±13.59
1038.38±8.29	36.90±2.26	38.80±6.35	69.9±7.89	69.5±11.17	44.50±7.3161.00±7.78	67.51±16.00	70.07±16.23
1236.35±8.53	37.21±3.28	37.09±6.63	59.33±5.87	63.25±6.68	41.29±6.7558.63±5.47	63.12±16.86	65.60±17.14
1434.52±8.85	35.43±2.29	35.71±6.81	60.86±5.84	62.86±6.21	35.20±4.1655.71±3.03	59.46±17.49	61.62±18.06
1632.90±9.12	32.87±2.60	34.57±6.90	54.47±3.19	56.97±2.95	33.16±3.7750.88±2.64	56.24±18.02	58.18±18.73
平均值 39.27±8.32	39.50±4.47	39.91±6.40	69.92±8.07	70.41±9.05	39.90±6.2361.95±6.00	69.42±14.95	72.01±14.65

K	CNMF	SGCNMF(α=0)	MFFS	HNMF	GNLMF	NMFLCAG	SCCNMF	GCNMF	SGCNMF
4	20.71±10.08	26.25±9.70	27.79±1.85	83.67±11.07	77.96±10.50	54.24±7.96	64.43±14.98	77.37±13.07	81.01±11.99
6	21.72±8.91	21.61±9.47	28.41±1.93	73.91±10.49	78.70±7.41	52.94+6.15	60.24±8.76	72.32±12.76	76.33±11.85
8	22.80±7.71	21.49±7.99	29.35±2.19	70.57±7.25	68.19±10.78	44.10±8.92	55.97±7.79	68.07±12.69	71.41±12.68
10	23.22±6.94	22.04±5.28	30.14±2.41	69.89±5.17	71.81±7.92	38.74±5.49	53.29±8.23	64.33±12.93	66.97±13.57
12	23.58±6.42	29.06±5.41	30.96±2.75	59.61±4.53	68.93±4.04	34.90±5.73	50.03±5.30	61.94±12.57	64.21±13.43
14	23.82±5.97	26.34±2.69	31.72±3.05	58.82±3.89	59.58±3.94	29.81±2.53	48.51±4.50	59.99±12.32	61.86±13.37
16	23.91±5.61	24.94±3.35	32.43±3.33	58.95±2.42	57.16±2.71	27.41+2.10	40.80±3.77	58.20±12.24	59.82±13.36
平均值	22.82±7.38	24.53±6.27	30.11±2.50	67.92±6.40	68.90±6.76	40.31±5.55	53.32±7.62	66.03±12.65	68.80±12.89

K	CNMF	SGCNMF(α=0)	MFFS	HNMF	GNLMF	NMFLCAG	SCCNMF	GCNMF	SGCNMF
4	41.37±3.78	50.50±8.13	47.56±1.96	64.75±12.06	65.88±14.74	56.64±7.81	56.13±4.42	61.00±8.67	63.13±10.97
6	38.79±4.63	44.75±9.93	43.53±4.45	54.33±9.65	53.67±9.88	49.58±3.26	50.83±4.71	55.85±10.32	57.75±11.38
8	35.94±5.81	37.94±6.66	40.92±5.24	51.31±9.66	50.44±7.38	38.69±10.12	52.69±11.82	51.78±10.88	53.22±11.94
10	33.82±6.35	35.15±4.25	38.78±5.88	47.40±9.93	49.50±7.71	36.75±8.24	47.25±3.18	48.21±11.44	49.23±12.58
12	32.02±6.79	32.17±3.26	37.08±6.28	44.40±5.24	43.96±7.13	35.24±4.61	39.00±8.25	45.05±12.14	45.98±13.09
14	30.36±7.26	29.54±3.62	35.74±6.48	39.75±4.77	39.50±7.95	28.21±10.57	37.79±3.54	42.37±12.62	43.17±13.53
16	29.09±7.44	28.22±2.79	34.58±6.64	42.91±4.88	42.88±7.02	29.46±3.53	35.59±2.66	40.12±12.96	40.87±13.77
平均值	34.48±6.01	36.90±5.52	39.74±5.28	49.26±8.03	49.40±8.83	39.22±6.16	45.61±5.51	49.20±11.29	50.48±12.47

K	CNMF	SGCNMF(α=0)	MFFS	HNMF	GNLMF	NMFLCAG	SCCNMF	GCNMF	SGCNMF
4	12.42±4.56	24.07±11.22	16.73±1.77	54.39±9.83	50.63±17.79	38.91±10.24	37.76±7.58	42.57±12.13	45.98±14.18
6	15.20±6.56	15.03±6.47	18.06±2.18	42.36±11.51	41.81±11.00	32.54±7.96	34.62±4.31	42.02±11.33	44.54±12.57
8	16.10±6.21	16.82±5.84	19.21±2.55	40.02±10.05	38.50±10.01	26.87±5.39	42.32±12.19	40.63±11.08	42.48±11.88
10	17.14±6.13	19.3±5.41	20.16±2.83	38.92±9.47	40.90±7.89	21.61±8.52	39.14±1.47	39.49±10.14	40.91±10.85
12	17.76±5.80	18.92±2.65	20.91±2.97	37.94±4.78	36.57±8.75	24.31±1.79	36.55±13.07	38.33±9.65	39.52±10.32
14	18.22±5.50	20.39±3.73	21.70±3.25	34.10±5.13	34.17±9.09	20.78±6.97	34.89±3.43	37.20±9.22	38.27±9.90
16	18.75±5.32	20.84±1.77	22.41±3.48	39.82±4.08	39.50±8.26	22.74±4.10	38.71±1.06	36.57±8.80	37.52±9.46
平均值	16.51±5.73	19.34±5.30	19.88±3.13	41.08±7.84	40.30±10.40	26.82±6.42	37.71±6.16	39.54±10.34	41.32±11.31

K	CNMF	SGCNMF	MFFS	SCCNMF	GCNMF	SGCNMF
2	92.48±6.24	91.25±11.57	94.59±4.81	92.62±10.98	98.83±1.50	98.83±1.50
3	86.67±7.60	87.91±7.44	84.79±10.35	91.63±8.29	96.08±4.11	96.27±3.96
4	77.94±9.25	73.00±12.84	73.03±12.41	84.92±10.91	89.51±8.35	89.56±8.20
5	72.72±9.40	66.77±8.05	67.30±9.13	82.26±7.29	89.62±10.23	90.57±10.66
6	59.09±7.64	65.82±6.66	65.13±8.21	77.01±8.36	85.80±7.75	88.50±7.33
7	58.20±6.77	59.42±6.37	59.91±6.92	70.47±6.89	82.04±5.52	84.36±6.15
8	54.38±3.60	59.62±7.81	58.37±4.84	72.51±11.43	77.97±9.43	80.33±8.97
9	49.58±3.37	53.27±6.77	56.72±5.11	66.60±5.11	74.67±7.58	76.79±7.26
10	47.5	54.26	51.76	63.34	73.9	77.35
平均值	66.51±6.73	67.92±8.44	61.75±7.77	77.93±8.66	85.38±6.81	86.95±6.75

Sparsity induced convex nonnegative matrix factorization algorithm with manifold regularization

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 35

Related Articles 7

Metrics

Recommended 0

[1]	Yuan XIE, Tao1 ZOU, Weijun SUN, Shengli XIE. Algorithm of underdetermined convolutive blind source separation for high reverberation environment [J]. Journal on Communications, 2023, 44(2): 82-93.
[2]	Huiqin WANG, Wenbin HOU, Qingbin PENG, Minghua CAO, Rui HUANG, Ling LIU. Step-by-step classification detection algorithm of SPPM based on K-means clustering [J]. Journal on Communications, 2022, 43(1): 161-171.
[3]	Xin JIN,Liang YANG,Cheng-ming JIN,Guo-hua SU,Lei SUN. Method to create and optimize original electric power communication network based on K-means [J]. Journal on Communications, 2016, 37(Z1): 10-14.
[4]	Hong-cheng LI,Xiao-ping WU,Yan CHEN. k-means clustering method preserving differential privacy in MapReduce framework [J]. Journal on Communications, 2016, 37(2): 125-131.
[5]	Dan QU,Wen-lin ZHANG. Sparse group LASSO constraint eigenphone speaker adaptation method for speech recognition [J]. Journal on Communications, 2015, 36(9): 47-54.
[6]	. Research on distributed genetic k-means for anomaly detection in MANET [J]. Journal on Communications, 2015, 36(11): 167-173.
[7]	Yan SONG,An-an LIU,Yong-dong ZHANG,Shou-xun LIN. Video text extraction method based on clustering [J]. Journal on Communications, 2009, 30(2): 138-142.

K	CNMF	SGCNMF	MFFS	SCCNMF	GCNMF	SGCNMF
2	66.82±19.03	66.96±28.02	74.13±17.45	81.70±11.92	92.11±8.68	92.11±8.68
3	65.30±11.71	66.03±14.29	62.56±15.14	81.34±9.54	87.08±10.07	87.44±9.74
4	61.93±9.68	55.60±14.43	55.42±12.51	72.88±7.31	80.40±8.76	80.40±8.67
5	55.96±9.30	49.93±6.87	52.14±8.42	71.70±9.16	82.72±8.89	84.35±9.57
6	47.65±7.70	53.05±6.25	53.21±7.22	69.25±7.18	79.55±5.53	82.17±4.86
7	48.36±5.40	49.00±8.12	51.12±5.38	59.50±6.56	75.01±5.39	78.04±5.31
8	47.40±4.85	50.60±6.16	50.24±3.78	68.13±10.70	75.38±6.22	76.50±6.02
9	43.93±3.80	45.01±5.14	49.49±3.9	65.00±3.42	73.05±5.65	75.88±3.67
10	42.61	47.81	47.02	64.22	76.59	77.93
平均值	53.33±8.93	53.77±11.16	51.23±6.87	70.41±8.22	80.21±7.40	81.65±7.07