联合低秩重构和投影重构的稳健特征选择方法

doi:10.11959/j.issn.1000-436x.2023061

通信学报 ›› 2023, Vol. 44 ›› Issue (3): 209-219.doi: 10.11959/j.issn.1000-436x.2023061

联合低秩重构和投影重构的稳健特征选择方法

仪双燕¹, 梁永生², 陆晶晶³, 柳伟⁴, 胡涛⁵, 何震宇⁶

¹ 深圳信息职业技术学院软件学院，广东深圳 518000
² 哈尔滨工业大学（深圳）电子与信息工程学院，广东深圳 518000
³ 深圳国微福芯技术有限公司，广东深圳 518000
⁴ 深圳信息职业技术学院计算机学院，广东深圳 518000
⁵ 深圳信息职业技术学院信息技术研究所，广东深圳 518000
⁶ 哈尔滨工业大学（深圳）计算机科学与技术学院，广东深圳 518000

修回日期:2023-02-02 出版日期:2023-03-25 发布日期:2023-03-01
作者简介:仪双燕（1987- ），女，山东菏泽人，博士，深圳信息职业技术学院讲师，主要研究方向为模式识别、机器学习
梁永生（1971- ），男，黑龙江肇东人，博士，哈尔滨工业大学教授，主要研究方向为通信信源、信道、网络协同优化编码
陆晶晶（1996- ），女，广西南宁人，深圳国微福芯技术有限公司工程师，主要研究方向为模式识别
柳伟（1973- ），男，湖南长沙人，博士，深圳信息职业技术学院教授，主要研究方向为人工智能、视觉媒体处理
胡涛（1979- ），男，湖北黄冈人，博士，深圳信息职业技术学院高级工程师，主要研究方向为图像处理、机器视觉等
何震宇（1978- ），男，江西抚州人，博士，哈尔滨工业大学（深圳）教授、博士生导师，主要研究方向为人工智能、计算机视觉等
基金资助:
国家自然科学基金资助项目(61906124);国家自然科学基金资助项目(62031013);中国博士后科学基金资助项目(2018M630158);广东省自然科学基金资助项目(2022A1515011447)

Robust feature selection method via joint low-rank reconstruction and projection reconstruction

Shuangyan YI¹, Yongsheng LIANG², Jingjing LU³, Wei LIU⁴, Tao HU⁵, Zhenyu HE⁶

¹ School of Software Engineering, Shenzhen Institute of Information Technology, Shenzhen 518000, China
² School of Electronics and Information Engineering, Harbin Institute of Technology (Shenzhen), Shenzhen 518000, China
³ Shenzhen Guowei Fuxin Technology Co., Ltd，Shenzhen 518000, China
⁴ School of Computer Sciences, Shenzhen Institute of Information Technology, Shenzhen 518000, China
⁵ Institute of Information Technology, Shenzhen Institute of Information Technology, Shenzhen 518000, China
⁶ School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen 518000, China

Revised:2023-02-02 Online:2023-03-25 Published:2023-03-01
Supported by:
The National Natural Science Foundation of China(61906124);The National Natural Science Foundation of China(62031013);The Chinese Postdoctoral Science Foundation(2018M630158);Guangdong Natural Science Foundation(2022A1515011447)

摘要/Abstract

摘要：

针对当前特征选择方法依然受噪声影响以及无法将聚类效果和重构效果有效统一的问题，提出了一种稳健的特征选择方法。从干净数据和重构数据作差的思路着手，将低秩重构数据和投影重构数据作差构建稳健的重构误差项，并提出从学习到的干净数据上选择特征用于聚类。将干净数据的学习和特征选择技能进行联合学习，相互促进，从而提升方法在有噪数据上的稳健性，并且将重构效果和聚类效果进行有效统一。在5个数据集上与几种图嵌入角度的特征选择以及PCA重构角度的特征选择方法进行聚类实验对比，实验结果表明，除LUNG噪声数据集外，所提方法在2种评价指标（ACC和NMI）下都优于对比特征选择方法。

关键词: 重构, 低秩, 投影, 稀疏, 特征选择

Abstract:

Aiming at the problem that current feature selection methods were still affected by noise and cannot effectively unify clustering and reconstruction effects, a robust feature selection method was proposed.A robust reconstruction error term was built by making the difference between low-rank reconstruction and projection reconstruction.After that, the features for clustering were selected from the reconstructed data instead of the original data.The learning of clean data and feature selection technique are allowed for joint learning and promote each other, thereby improving the robustness of the method on noisy data, and effectively unifying reconstruction and clustering.Compared with several kinds of graph embedding feature selection and reconstruction feature selection methods on five datasets, the experimental results showed that, except for the LUNG noise dataset, the proposed method outperforms the comparative feature selection method under both evaluation indicators (ACC and NMI).

Key words: reconstruction, low-rank, projection, sparsity, feature selection

中图分类号:

TN92

仪双燕, 梁永生, 陆晶晶, 柳伟, 胡涛, 何震宇. 联合低秩重构和投影重构的稳健特征选择方法[J]. 通信学报, 2023, 44(3): 209-219.

Shuangyan YI, Yongsheng LIANG, Jingjing LU, Wei LIU, Tao HU, Zhenyu HE. Robust feature selection method via joint low-rank reconstruction and projection reconstruction[J]. Journal on Communications, 2023, 44(3): 209-219.

图/表 8

图1

表1

表2

不同方法在不同数据集上的聚类ACC"

数据集	噪声	BaseLine	UDFS	RUFS	SOGFS	AW-SPCA	SPCA	USFS	FOG	所提方法
ORL	—	0.518 1	0.525 9	0.540 4	0.533 3	0.524 3	0.535 9	0.537 9	0.534 5	$0 . 5548$
	20%	0.359 1	0.374 5	0.377 1	0.368 4	0.372 1	0.367 3	0.378 9	0.429 3	$0 . 4439$
MNIST	—	0.423 2	0.460 3	0.451 4	0.456 0	0.492 4	0.457 2	0.522 6	0.501 7	$0 . 5591$
	20%	0.379 5	0.385 8	0.400 1	0.390 2	0.424 1	0.425 4	0.419 2	0.386 5	$0 . 4556$
COIL20	—	0.539 3	0.563 6	0.584 3	0.567 9	0.574 2	0.584 3	0.613 3	0.586 9	$0 . 6310$
	20%	0.400 2	0.423 8	0.450 0	0.435 5	0.494 7	0.469 7	0.485 8	0.455 9	$0 . 6284$
USPS	—	0.566 8	0.591 0	0.588 7	0.589 9	0.655 9	0.624 7	0.651 6	0.613 3	$0 . 6711$
	20%	0.496 1	0.519 6	0.523 1	0.528 6	0.533 9	0.528 6	0.536 4	0.529 7	$0 . 5411$
LUNG	—	0.730 3	0.718 0	0.759 9	0.714 0	0.743 3	0.744 8	0.761 3	0.752 5	$0 . 7887$
	20%	0.568 2	0.625 6	0.651 7	0.616 3	0.641 6	0.641 1	$0 . 6571$	0.636 9	0.645 3

表2

表3

不同方法在不同数据集上的NMI"

数据集	噪声	BaseLine	UDFS	RUFS	AW-SPCA	SPCA	USFS	FOG	所提方法
ORL	—	0.718 7	0.727 4	0.737 3	0.727 5	0.731 2	0.732 5	0.731 4	$0 . 7468$
	20%	0.573 0	0.589 0	0.590 0	0.588 0	0.589 0	0.595 0	0.581 2	$0 . 5953$
MNIST	—	0.338 3	0.391 2	0.377 0	0.421 5	0.382 9	0.449 2	0.435 9	$0 . 4965$
	20%	0.282 0	0.291 0	0.304 0	0.332 0	0.344 0	0.324 0	0.290 6	$0 . 3672$
COIL20	—	0.709 3	0.721 5	0.735 1	0.737 5	0.723 4	0.747 3	0.729 1	$0 . 7514$
	20%	0.522 0	0.546 0	0.571 0	0.613 0	0.597 0	0.606 0	0.563 6	$0 . 7478$
USPS	—	0.546 0	0.552 7	0.559 0	0.611 6	0.579 3	0.610 7	0.573 0	$0 . 6142$
	20%	0.446 0	0.464 0	0.458 0	0.482 0	0.470 0	0.474 0	0.461 6	$0 . 4842$
LUNG	—	0.521 4	0.518 2	0.543 5	0.538 7	0.539 1	0.545 8	0.539 4	$0 . 5601$
	20%	0.267 0	0.337 0	0.344 0	0.327 0	0.327 0	$0 . 3510$	0.334 5	0.344 7

表3

图2

图3

图4

图5

参考文献 42

[1]	JENNIFER G D , BRODLEY C E . Feature selection for unsupervised learning[J]. Journal of Machine Learning Research, 2004,5: 845-889.
[2]	HE X F , CAI D , PARTHA N . Laplacian score for feature selection[C]// Advances in Neural Information Processing Systems. Massachusetts:MIT Press, 2006: 507-514.
[3]	ZHAO Z , LIU H . Spectral feature selection for supervised and unsupervised learning[C]// Proceedings of the 24th International Conference on Machine Learning. New York:ACM Press, 2007: 1151-1157.
[4]	NIE F P , XIANG S M , JIA Y Q ,et al. Trace ratio criterion for feature selection[C]// Proceedings of the 23rd National Conference on Artificial Intelligence. New York:ACM Press, 2008: 671-676.
[5]	ZHAO Z , WANG L , LIU H . Efficient spectral feature selection with minimum redundancy[C]// Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence. New York:ACM Press, 2010: 673-678.
[6]	LI Z C , YANG Y , LIU J ,et al. Unsupervised feature selection using nonnegative spectral analysis[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021,26(1): 1026-1032.
[7]	ZHU P F , ZUO W M , ZHANG L ,et al. Unsupervised feature selection by regularized self-representation[J]. Pattern Recognition, 2015,48(2): 438-446.
[8]	LEI C , ZHU X F . Unsupervised feature selection via local structure learning and sparse learning[J]. Multimedia Tools and Applications, 2018,77(22): 29605-29622.
[9]	ARGYRIOU A , EVGENIOU T , PONTIL M . Convex multi-task feature learning[J]. Machine Learning, 2008,73(3): 243-272.
[10]	刘文慧 . PCA与PLS用于高维数据分类的比较性研究[C]// 2011年中国卫生统计学年会论文集.[S.l.:s.n.], 2011: 422-424.
	LIU W H , . Comparative study of PCA and PLS for high-dimensional data classification[C]// Proceedings of the 2011 China Annual Conference of Health Statistics.[S.l.:s.n.], 2011: 422-424.
[11]	WANG F , ZHU L , LI J ,et al. Unsupervised soft-label feature selection[J]. Knowledge-Based Systems, 2021:doi.osrg/10.1016/j.knosys.2021.106847.
[12]	CAI D , ZHANG C Y , HE X F . Unsupervised feature selection for multi-cluster data[C]// Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2010: 25-28.
[13]	YANG Y , SHEN H T , MA Z G ,et al. l_2,1-norm regularized discriminative feature selection for unsupervised learning[C]// Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence. New York:ACM Press, 2011: 1589-1594.
[14]	SHI Y , MIAO J Y , WANG Z Y ,et al. Feature selection with l_2,1-2regularization[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018,29(10): 4967-4982.
[15]	QIAN M , ZHAI C . Robust unsupervised feature selection[C]// Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. New York:ACM Press, 2013: 1621-1627.
[16]	SHI L , DU L , SHEN Y D . Robust spectral learning for unsupervised feature selection[C]// Proceedings of 2014 IEEE International Conference on Data Mining. Piscataway:IEEE Press, 2015: 977-982.
[17]	NIE F P , ZHU W , LI X L . Unsupervised feature selection with structured graph optimization[C]// Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. New York:ACM Press, 2016: 1302-1308.
[18]	CHEN H , NIE F P , WANG R ,et al. Unsupervised feature selection with flexible optimal graph[J]. IEEE Transactions on Neural Networks and Learning Systems， 2022:doi.org/10.1109/TNNLS.2022.3186171.
[19]	TURK M A , PENTLAND A P . Face recognition using eigenfaces[C]// Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 1991: 586-591.
[20]	ZOU H , HASTIE T , TIBSHIRANI R . Sparse principal component analysis[J]. Journal of Computational and Graphical Statistics, 2006,15(2): 265-286.
[21]	SEGHOUANE A K , SHOKOUHI N , KOCH I . Sparse principal component analysis with preserved sparsity pattern[J]. IEEE Transactions on Image Processing, 2019,28(7): 3274-3285.
[22]	YI S Y , HE Z Y , JING X Y ,et al. Adaptive weighted sparse principal component analysis for robust unsupervised feature selection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020,31(6): 2153-2163.
[23]	CANDES E J , LI X , MA Y . Robust principal component analysis?[J]. Journal of the ACM, 2011,58(3): 1-39.
[24]	WRIGHT J , PENG Y , MA Y ,et al. Robust principal component analysis:exact recovery of corrupted low-rank matrices via convex optimization[C]// Proceedings of Neural Information Processing Systems. Massachusetts:MIT Press, 2009: 1-9.
[25]	LIU G C , LIN Z C , YU Y . Robust subspace segmentation by low-rank representation[C]// Proceedings of the 27th International Conference on International Conference on Machine Learning. New York:ACM Press, 2010: 663-670.
[26]	LIU G C , LIN Z C , YAN S C ,et al. Robust recovery of subspace structures by low-rank representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013,35(1): 171-184.
[27]	SHAHID N , KALOFOLIAS V , BRESSON X ,et al. Robust principal component analysis on graphs[C]// Proceedings of 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2016: 2812-2820.
[28]	NIE F P , YUAN J J , HUANG H . Optimal mean robust principal component analysis[C]// Proceedings of the 31st International Conference on International Conference on Machine Learning. New York:ACM Press, 2014: 1062-1070.
[29]	SHI X S , NIE F P , LAI Z H ,et al. Robust principal component analysis via optimal mean by joint l_2,1and Schatten p-norms minimization[J]. Neurocomputing, 2018,283: 205-213.
[30]	LUO M N , NIE F P , CHANG X J ,et al. Avoiding optimal mean l_2,1-norm maximization-based robust PCA for reconstruction[J]. Neural Computation, 2017,29(4): 1124-1150.
[31]	LUO M N , NIE F P , CHANG X J ,et al. Avoiding optimal mean robust PCA/2DPCA with non-greedy l₁-norm maximization[C]// Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. New York:ACM Press, 2016: 1802-1808.
[32]	LIAO S L , LI J , LIU Y ,et al. Robust formulation for PCA:avoiding mean calculation with l_2,p-norm maximization[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2018: 3604-3610.
[33]	NIE F P , TIAN L , HUANG H ,et al. Non-greedy l₂₁-norm maximization for principal component analysis[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2021,30: 5277-5286.
[34]	WANG Q Q , GAO Q X , GAO X B ,et al. l_2,p-norm based PCA for image recognition[J]. IEEE Transactions on Image Processing:A Publication of the IEEE Signal Processing Society, 2018,27(3): 1336-1346.
[35]	NIE F P , WU D Y , WANG R ,et al. Truncated robust principle component analysis with a general optimization framework[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022,44(2): 1081-1097.
[36]	QW A , QXG A , GAN S B ,et al. Double robust principal component analysis[J]. Neurocomputing, 2020,391: 119-128.
[37]	NENE S A , NAYAR S K , MURASE H ,et al. Columbia object image library[R]. 1996.
[38]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.
[39]	HULL J J . A database for handwritten text recognition research[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994,16(5): 550-554.
[40]	SINGH D , FEBBO P G , ROSS K ,et al. Gene expression correlates of clinical prostate cancer behavior[J]. Cancer Cell, 2002,1(2): 203-209.
[41]	WANG R , BIAN J T , NIE F P ,et al. Unsupervised discriminative projection for feature selection[J]. IEEE Transactions on Knowledge and Data Engineering, 2022,34(2): 942-953.
[42]	HU H J , WANG R , NIE F P ,et al. Fast unsupervised feature selection with anchor graph and l_2,1-norm regularization[J]. Multimedia Tools and Applications, 2018,77(17): 22099-22113.

联合低秩重构和投影重构的稳健特征选择方法

Robust feature selection method via joint low-rank reconstruction and projection reconstruction

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 42

相关文章 15

Metrics

推荐阅读 0

数据集	样本数	特征数	类别数
ORL	400	1 024	40
COIL20	1 440	1 024	20
MNIST	2 000	784	10
USPS	9 298	256	10
LUNG	203	3 312	5

[1]	王忠勇, 李正豪, 巩克现, 孙鹏, 李清涛. 高误码率下基于随机抽取的LDPC码校验矩阵重建[J]. 通信学报, 2023, 44(3): 128-137.
[2]	崔伟, 于颖, 于海霞, 陈超, 李云鹏. 基于IOC-CSMP的OFDM系统稀疏信道快速重构算法[J]. 通信学报, 2023, 44(2): 52-58.
[3]	侯进, 陈鑫强. 基于几何序列分解与稀疏重构的DOA估计[J]. 通信学报, 2023, 44(1): 153-163.
[4]	赖恪, 雷菁, 刘伟, 文磊. 基于网络编码的稀疏码分多址接入HARQ方案[J]. 通信学报, 2022, 43(9): 57-69.
[5]	王鼎, 高卫港, 吴志东. 基于稀疏贝叶斯的阵列幅相和互耦误差联合校正方法[J]. 通信学报, 2022, 43(9): 112-120.
[6]	曾嵘, 杭潇. 车联网环境下可重构智能反射面辅助无线信道估计算法[J]. 通信学报, 2022, 43(8): 142-150.
[7]	郭海燕, 杨震, 邹玉龙, 吕斌, 冯蕴天, 赵玉娟. 基于主被动波束成形联合优化的双RIS辅助抗干扰通信方法[J]. 通信学报, 2022, 43(7): 21-30.
[8]	冯晓伟, 许剑锋, 何川. 动态广义主成分分析及其在故障子空间建模中的应用[J]. 通信学报, 2022, 43(5): 92-101.
[9]	张汝云, 肖戈扬, 单麒赫, 邹涛, 李丹, 滕菲. 多模态网络下多智能体协同控制的通信拓扑重构方法[J]. 通信学报, 2022, 43(4): 50-59.
[10]	雷拓峰, 倪淑燕, 程乃平, 宋鑫. 瑞利衰落信道下SCMA码本设计[J]. 通信学报, 2022, 43(4): 107-113.
[11]	尹安琪, 郭渊博, 汪定, 曲彤洲, 陈琳. 可证明安全的抗量子两服务器口令认证密钥交换协议[J]. 通信学报, 2022, 43(3): 14-29.
[12]	李斌, 周清雷, 陈晓杰, 冯峰. 可重构的素域SM2算法优化方法[J]. 通信学报, 2022, 43(3): 30-41.
[13]	周嵩林, 唐隽文, 刘罗颢, 吴优, 刘长昊, 金一飞, 杨帆, 许慎恒, 李懋坤. 基于电磁表面的阵列天线及应用概述[J]. 通信学报, 2022, 43(12): 13-23.
[14]	唐奎, 胡琪, 赵俊明, 陈克, 冯一军. 基于RIS的室内无线通信信号增强系统[J]. 通信学报, 2022, 43(12): 24-31.
[15]	刘海霞, 易浩, 马向进, 乐舒瑶, 孔旭东, 马培, 曾宇鑫, 李龙. 基于无源可重构智能超表面的室内无线信号覆盖增强[J]. 通信学报, 2022, 43(12): 32-44.