联合低秩重构和投影重构的稳健特征选择方法

doi:10.11959/j.issn.1000-436x.2023061

Abstract

Abstract:

Aiming at the problem that current feature selection methods were still affected by noise and cannot effectively unify clustering and reconstruction effects, a robust feature selection method was proposed.A robust reconstruction error term was built by making the difference between low-rank reconstruction and projection reconstruction.After that, the features for clustering were selected from the reconstructed data instead of the original data.The learning of clean data and feature selection technique are allowed for joint learning and promote each other, thereby improving the robustness of the method on noisy data, and effectively unifying reconstruction and clustering.Compared with several kinds of graph embedding feature selection and reconstruction feature selection methods on five datasets, the experimental results showed that, except for the LUNG noise dataset, the proposed method outperforms the comparative feature selection method under both evaluation indicators (ACC and NMI).

Key words: reconstruction, low-rank, projection, sparsity, feature selection

CLC Number:

TN92

Shuangyan YI, Yongsheng LIANG, Jingjing LU, Wei LIU, Tao HU, Zhenyu HE. Robust feature selection method via joint low-rank reconstruction and projection reconstruction[J]. Journal on Communications, 2023, 44(3): 209-219.

Figures/Tables 8

数据集	噪声	BaseLine	UDFS	RUFS	SOGFS	AW-SPCA	SPCA	USFS	FOG	所提方法
ORL	—	0.518 1	0.525 9	0.540 4	0.533 3	0.524 3	0.535 9	0.537 9	0.534 5	$0 . 5548$
	20%	0.359 1	0.374 5	0.377 1	0.368 4	0.372 1	0.367 3	0.378 9	0.429 3	$0 . 4439$
MNIST	—	0.423 2	0.460 3	0.451 4	0.456 0	0.492 4	0.457 2	0.522 6	0.501 7	$0 . 5591$
	20%	0.379 5	0.385 8	0.400 1	0.390 2	0.424 1	0.425 4	0.419 2	0.386 5	$0 . 4556$
COIL20	—	0.539 3	0.563 6	0.584 3	0.567 9	0.574 2	0.584 3	0.613 3	0.586 9	$0 . 6310$
	20%	0.400 2	0.423 8	0.450 0	0.435 5	0.494 7	0.469 7	0.485 8	0.455 9	$0 . 6284$
USPS	—	0.566 8	0.591 0	0.588 7	0.589 9	0.655 9	0.624 7	0.651 6	0.613 3	$0 . 6711$
	20%	0.496 1	0.519 6	0.523 1	0.528 6	0.533 9	0.528 6	0.536 4	0.529 7	$0 . 5411$
LUNG	—	0.730 3	0.718 0	0.759 9	0.714 0	0.743 3	0.744 8	0.761 3	0.752 5	$0 . 7887$
	20%	0.568 2	0.625 6	0.651 7	0.616 3	0.641 6	0.641 1	$0 . 6571$	0.636 9	0.645 3

数据集	噪声	BaseLine	UDFS	RUFS	AW-SPCA	SPCA	USFS	FOG	所提方法
ORL	—	0.718 7	0.727 4	0.737 3	0.727 5	0.731 2	0.732 5	0.731 4	$0 . 7468$
	20%	0.573 0	0.589 0	0.590 0	0.588 0	0.589 0	0.595 0	0.581 2	$0 . 5953$
MNIST	—	0.338 3	0.391 2	0.377 0	0.421 5	0.382 9	0.449 2	0.435 9	$0 . 4965$
	20%	0.282 0	0.291 0	0.304 0	0.332 0	0.344 0	0.324 0	0.290 6	$0 . 3672$
COIL20	—	0.709 3	0.721 5	0.735 1	0.737 5	0.723 4	0.747 3	0.729 1	$0 . 7514$
	20%	0.522 0	0.546 0	0.571 0	0.613 0	0.597 0	0.606 0	0.563 6	$0 . 7478$
USPS	—	0.546 0	0.552 7	0.559 0	0.611 6	0.579 3	0.610 7	0.573 0	$0 . 6142$
	20%	0.446 0	0.464 0	0.458 0	0.482 0	0.470 0	0.474 0	0.461 6	$0 . 4842$
LUNG	—	0.521 4	0.518 2	0.543 5	0.538 7	0.539 1	0.545 8	0.539 4	$0 . 5601$
	20%	0.267 0	0.337 0	0.344 0	0.327 0	0.327 0	$0 . 3510$	0.334 5	0.344 7

References 42

[1]	JENNIFER G D , BRODLEY C E . Feature selection for unsupervised learning[J]. Journal of Machine Learning Research, 2004,5: 845-889.
[2]	HE X F , CAI D , PARTHA N . Laplacian score for feature selection[C]// Advances in Neural Information Processing Systems. Massachusetts:MIT Press, 2006: 507-514.
[3]	ZHAO Z , LIU H . Spectral feature selection for supervised and unsupervised learning[C]// Proceedings of the 24th International Conference on Machine Learning. New York:ACM Press, 2007: 1151-1157.
[4]	NIE F P , XIANG S M , JIA Y Q ,et al. Trace ratio criterion for feature selection[C]// Proceedings of the 23rd National Conference on Artificial Intelligence. New York:ACM Press, 2008: 671-676.
[5]	ZHAO Z , WANG L , LIU H . Efficient spectral feature selection with minimum redundancy[C]// Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence. New York:ACM Press, 2010: 673-678.
[6]	LI Z C , YANG Y , LIU J ,et al. Unsupervised feature selection using nonnegative spectral analysis[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021,26(1): 1026-1032.
[7]	ZHU P F , ZUO W M , ZHANG L ,et al. Unsupervised feature selection by regularized self-representation[J]. Pattern Recognition, 2015,48(2): 438-446.
[8]	LEI C , ZHU X F . Unsupervised feature selection via local structure learning and sparse learning[J]. Multimedia Tools and Applications, 2018,77(22): 29605-29622.
[9]	ARGYRIOU A , EVGENIOU T , PONTIL M . Convex multi-task feature learning[J]. Machine Learning, 2008,73(3): 243-272.
[10]	刘文慧 . PCA与PLS用于高维数据分类的比较性研究[C]// 2011年中国卫生统计学年会论文集.[S.l.:s.n.], 2011: 422-424.
	LIU W H , . Comparative study of PCA and PLS for high-dimensional data classification[C]// Proceedings of the 2011 China Annual Conference of Health Statistics.[S.l.:s.n.], 2011: 422-424.
[11]	WANG F , ZHU L , LI J ,et al. Unsupervised soft-label feature selection[J]. Knowledge-Based Systems, 2021:doi.osrg/10.1016/j.knosys.2021.106847.
[12]	CAI D , ZHANG C Y , HE X F . Unsupervised feature selection for multi-cluster data[C]// Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2010: 25-28.
[13]	YANG Y , SHEN H T , MA Z G ,et al. l_2,1-norm regularized discriminative feature selection for unsupervised learning[C]// Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence. New York:ACM Press, 2011: 1589-1594.
[14]	SHI Y , MIAO J Y , WANG Z Y ,et al. Feature selection with l_2,1-2regularization[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018,29(10): 4967-4982.
[15]	QIAN M , ZHAI C . Robust unsupervised feature selection[C]// Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. New York:ACM Press, 2013: 1621-1627.
[16]	SHI L , DU L , SHEN Y D . Robust spectral learning for unsupervised feature selection[C]// Proceedings of 2014 IEEE International Conference on Data Mining. Piscataway:IEEE Press, 2015: 977-982.
[17]	NIE F P , ZHU W , LI X L . Unsupervised feature selection with structured graph optimization[C]// Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. New York:ACM Press, 2016: 1302-1308.
[18]	CHEN H , NIE F P , WANG R ,et al. Unsupervised feature selection with flexible optimal graph[J]. IEEE Transactions on Neural Networks and Learning Systems， 2022:doi.org/10.1109/TNNLS.2022.3186171.
[19]	TURK M A , PENTLAND A P . Face recognition using eigenfaces[C]// Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 1991: 586-591.
[20]	ZOU H , HASTIE T , TIBSHIRANI R . Sparse principal component analysis[J]. Journal of Computational and Graphical Statistics, 2006,15(2): 265-286.
[21]	SEGHOUANE A K , SHOKOUHI N , KOCH I . Sparse principal component analysis with preserved sparsity pattern[J]. IEEE Transactions on Image Processing, 2019,28(7): 3274-3285.
[22]	YI S Y , HE Z Y , JING X Y ,et al. Adaptive weighted sparse principal component analysis for robust unsupervised feature selection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020,31(6): 2153-2163.
[23]	CANDES E J , LI X , MA Y . Robust principal component analysis?[J]. Journal of the ACM, 2011,58(3): 1-39.
[24]	WRIGHT J , PENG Y , MA Y ,et al. Robust principal component analysis:exact recovery of corrupted low-rank matrices via convex optimization[C]// Proceedings of Neural Information Processing Systems. Massachusetts:MIT Press, 2009: 1-9.
[25]	LIU G C , LIN Z C , YU Y . Robust subspace segmentation by low-rank representation[C]// Proceedings of the 27th International Conference on International Conference on Machine Learning. New York:ACM Press, 2010: 663-670.
[26]	LIU G C , LIN Z C , YAN S C ,et al. Robust recovery of subspace structures by low-rank representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013,35(1): 171-184.
[27]	SHAHID N , KALOFOLIAS V , BRESSON X ,et al. Robust principal component analysis on graphs[C]// Proceedings of 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2016: 2812-2820.
[28]	NIE F P , YUAN J J , HUANG H . Optimal mean robust principal component analysis[C]// Proceedings of the 31st International Conference on International Conference on Machine Learning. New York:ACM Press, 2014: 1062-1070.
[29]	SHI X S , NIE F P , LAI Z H ,et al. Robust principal component analysis via optimal mean by joint l_2,1and Schatten p-norms minimization[J]. Neurocomputing, 2018,283: 205-213.
[30]	LUO M N , NIE F P , CHANG X J ,et al. Avoiding optimal mean l_2,1-norm maximization-based robust PCA for reconstruction[J]. Neural Computation, 2017,29(4): 1124-1150.
[31]	LUO M N , NIE F P , CHANG X J ,et al. Avoiding optimal mean robust PCA/2DPCA with non-greedy l₁-norm maximization[C]// Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. New York:ACM Press, 2016: 1802-1808.
[32]	LIAO S L , LI J , LIU Y ,et al. Robust formulation for PCA:avoiding mean calculation with l_2,p-norm maximization[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2018: 3604-3610.
[33]	NIE F P , TIAN L , HUANG H ,et al. Non-greedy l₂₁-norm maximization for principal component analysis[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2021,30: 5277-5286.
[34]	WANG Q Q , GAO Q X , GAO X B ,et al. l_2,p-norm based PCA for image recognition[J]. IEEE Transactions on Image Processing:A Publication of the IEEE Signal Processing Society, 2018,27(3): 1336-1346.
[35]	NIE F P , WU D Y , WANG R ,et al. Truncated robust principle component analysis with a general optimization framework[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022,44(2): 1081-1097.
[36]	QW A , QXG A , GAN S B ,et al. Double robust principal component analysis[J]. Neurocomputing, 2020,391: 119-128.
[37]	NENE S A , NAYAR S K , MURASE H ,et al. Columbia object image library[R]. 1996.
[38]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.
[39]	HULL J J . A database for handwritten text recognition research[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994,16(5): 550-554.
[40]	SINGH D , FEBBO P G , ROSS K ,et al. Gene expression correlates of clinical prostate cancer behavior[J]. Cancer Cell, 2002,1(2): 203-209.
[41]	WANG R , BIAN J T , NIE F P ,et al. Unsupervised discriminative projection for feature selection[J]. IEEE Transactions on Knowledge and Data Engineering, 2022,34(2): 942-953.
[42]	HU H J , WANG R , NIE F P ,et al. Fast unsupervised feature selection with anchor graph and l_2,1-norm regularization[J]. Multimedia Tools and Applications, 2018,77(17): 22099-22113.

Metrics

Recommended 0

No Suggested Reading articles found!

Robust feature selection method via joint low-rank reconstruction and projection reconstruction

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 42

Related Articles 15

Metrics

Recommended 0

数据集	样本数	特征数	类别数
ORL	400	1 024	40
COIL20	1 440	1 024	20
MNIST	2 000	784	10
USPS	9 298	256	10
LUNG	203	3 312	5

[1]	Wei CUI, Ying YU, Haixia YU, Chao CHEN, Yunpeng LI. Sparse channel fast reconstruction algorithm for OFDM system based on IOC-CSMP [J]. Journal on Communications, 2023, 44(2): 52-58.
[2]	Jin HOU, Xinqiang CHEN. DOA estimation based on geometric sequence decomposition and sparse reconstruction [J]. Journal on Communications, 2023, 44(1): 153-163.
[3]	Ding WANG, Weigang GAO, Zhidong WU. Array amplitude-phase and mutual coupling error joint correction method based on sparse Bayesian [J]. Journal on Communications, 2022, 43(9): 112-120.
[4]	Yanwen WANG, Weimin LEI, Wei ZHANG, Huan MENG, Xinyi CHEN, Wenhui YE, Qingyang JING. Survey on video image reconstruction method based on generative model [J]. Journal on Communications, 2022, 43(9): 194-208.
[5]	Ang LI, Jianxin CHEN, Xin WEI, Liang ZHOU. 6G-oriented cross-modal signal reconstruction technology [J]. Journal on Communications, 2022, 43(6): 28-40.
[6]	Xiaofeng FENG, Jianfeng XU, Chuan HE. Dynamic generalized principal component analysis with applications to fault subspace modeling [J]. Journal on Communications, 2022, 43(5): 92-101.
[7]	Haixia LIU, Hao YI, Xiangjin MA, Shuyao YUE, Xudong KONG, Pei MA, Yuxin ZENG, Long LI. Indoor wireless signal coverage and enhancement based on passive reconfigurable intelligent metasurface [J]. Journal on Communications, 2022, 43(12): 32-44.
[8]	Bin YU, Nan ZHANG, Xu LU, Zhenhua DUAN, Cong TIAN. Runtime verification approach for DoS attack detection in edge servers [J]. Journal on Communications, 2021, 42(9): 75-86.
[9]	Fangfang GUO, Hongwu LYU, Weilin REN, Ruini WANG. Reduction algorithm based on supervised discriminant projection for network security data [J]. Journal on Communications, 2021, 42(6): 84-93.
[10]	Zhaojun WU, Limin ZHANG, Zhaogen ZHONG, Renxin LIU. Reconstruction of sparse check matrix for LDPC at high bit error rate [J]. Journal on Communications, 2021, 42(3): 1-10.
[11]	Zhengquan LI, Yuan LIN, Mengya LI, Yang LIU, Qiong WU, Song XING. Digital modulation recognition based on discriminative restricted Boltzmann machine [J]. Journal on Communications, 2021, 42(2): 81-91.
[12]	Hai LIU, Youliang TIAN, Ying TANG, Jianbing Ni, Jianfeng MA. Design models of secret reconstruction towards rational users [J]. Journal on Communications, 2021, 42(11): 54-65.
[13]	Jin HOU, Yunzhe LI, Tianyu LI. Underdetermined DOA estimation of coherent signals based on denoising complex FastICA and sparse reconstruction [J]. Journal on Communications, 2021, 42(11): 172-181.
[14]	Qizhen ZHOU,Jianchun XING,Qiliang YANG,Deshuai HAN. Sequential image deep learning-based Wi-Fi human activity recognition method [J]. Journal on Communications, 2020, 41(8): 43-54.
[15]	Yonghao LI, Liang HU, Ping ZHANG, Wanfu GAO. Multi-label feature selection based on dynamic graph Laplacian [J]. Journal on Communications, 2020, 41(12): 47-59.