基于K-means聚类的子结构相关适配迁移学习方法

doi:10.11959/j.issn.1000-0801.2023045

Abstract

Abstract:

Domain drifts severely affect the performance of traditional machine learning methods, and existing domain adaptive methods are mainly represented by adaptive adjustment cross-domain through global, class-level, or sample-level distribution adaptation.However, too coarse global matching and class-level matching can lead to insufficient adaptation, and sample-level adaptation to noise can lead to excessive adaptation.A substructure correlation adaptation (SCOAD) transfer learning algorithm based on K-means clustering was proposed.Firstly, multiple subdomains of the source domain and the target domain were obtained by K-means clustering.Then, the matching of the second-order statistics of the subdomain center was sought.Finally, the target domain samples were classified by using the subdomain structure.The proposed method approach further improves the performance of knowledge transfer between the source and target domains on top of the traditional approach.Experimental results on common transfer learning datasets show the effectiveness of the proposed method.

Key words: transfer learning, domain adaptation, substructural adaptation, clustering

Haoshuang LIU, Yong ZHANG, Yingbo CAO. Substructure correlation adaptation transfer learning method based on K-means clustering[J]. Telecommunications Science, 2023, 39(3): 124-134.

Figures/Tables 9

数据集	1-NN	TCA	TJM	OTDA	CORAL	EASYTL	SCOAD
MNIST→USPS	65.9%	54.3%	42.9%	49.3%	65.6%	54.5%	$66 . 2 %$
USPS→MNIST	44.7%	$52 . 2 %$	41.4%	50.7%	46.9%	31.0%	51.2%
Caltech→Amazon	23.7%	43.4%	46.7%	30.5%	47.2%	$50 . 1 %$	40.0%
Caltech→Webcam	25.8%	37.3%	38.0%	23.8%	39.2%	$49 . 5 %$	42.0%
Caltech→DSLR	25.5%	44.0%	43.3%	26.6%	40.7%	48.4%	$50 . 3 %$
Amazon→Caltech	26.0%	38.2%	39.6%	29.4%	40.3%	$43 . 0 %$	38.6%
Amazon→Webcam	29.8%	38.0%	$40 . 7 %$	25.6%	38.7%	$40 . 7 %$	$40 . 7 %$
Amazon→DSLR	25.5%	30.6%	42.2%	35.7%	38.3%	38.9%	$42 . 7 %$
Webcam→Caltech	19.9%	29.7%	30.2%	25.9%	34.6%	29.7%	$35 . 4 %$
Webcam→Amazon	23.0%	32.3%	29.9%	27.4%	37.8%	35.2%	$40 . 2 %$
Webcam→DSLR	59.2%	82.4%	$88 . 7 %$	76.5%	84.9%	77.1%	75.8%
DSLR→Caltech	26.3%	30.9%	31.5%	27.3%	34.2%	31.2%	$34 . 3 %$
DSLR→Amazon	28.5%	29.3%	32.6%	29.1%	$38 . 1 %$	31.9%	36.5%
DSLR→Webcam	63.4%	84.8%	85.6%	65.7%	85.9%	69.5%	$86 . 4 %$
平均值	34.8%	44.8%	45.2%	37.4%	48.0%	45.0%	$48 . 6 %$

数据集	1-NN	TCA	BDA	JGSA	CORAL	SCOAD
Books→DVDs	49.6%	63.6%	64.2%	66.6%	$71 . 6 %$	65.4%
Books→Electronics	49.8%	60.9%	62.1%	$75 . 0 %$	65.1%	69.2%
Books→Kitchen	50.3%	64.2%	65.4%	72.1%	67.3%	$74 . 0 %$
DVDs→Books	53.3%	63.3%	62.4%	55.5%	70.1%	$72 . 4 %$
DVDs→Electronics	51.0%	64.2%	66.3%	67.3%	65.6%	$70 . 0 %$
DVDs→Kitchen	53.1%	69.1%	68.9%	65.6%	67.1%	$70 . 0 %$
Electronics→Books	50.8%	59.5%	59.2%	51.6%	67.1%	$69 . 0 %$
Electronics→DVDs	50.9%	62.1%	61.6%	50.8%	66.2%	$67 . 6 %$
Electronics→Kitchen	51.2%	74.8%	74.7%	55.0%	77.6%	$81 . 0 %$
Kitchen→Books	52.2%	64.1%	62.7%	58.3%	68.2%	$69 . 1 %$
Kitchen→DVDs	51.2%	65.4%	64.3%	56.4%	68.9%	$69 . 6 %$
Kitchen→Electronics	52.3%	74.5%	74.0%	51.7%	75.4%	$78 . 1 %$
平均值	51.3%	65.5%	65.5%	60.5%	69.2%	$71 . 7 %$

算法	1-NN	TCA	TJM	OTDA	CORAL	EASYTL	SCOAD
序值	6.35	4.25	3.50	5.78	2.71	3.24	$2 . 14$

算法	1-NN	TCA	BDA	JGSA	CORAL	SCOAD
序值	5.83	3.58	3.83	4.08	2.42	$1 . 25$

References 34

[1]	ZHUANG F Z , QI Z Y , DUAN K Y ,et al. A comprehensive survey on transfer learning[J]. Proceedings of the IEEE, 2021,109(1): 43-76.
[2]	GRETTON A , BORGWARDT K M , RASCH M J ,et al. A kernel two-sample test[J]. The Journal of Machine Learning Research, 2012,13(1): 723-773.
[3]	SUN B C , FENG J S , SAENKO K . Return of frustratingly easy domain adaptation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2016,30(1): 2058-2065.
[4]	GAO J , HUANG R , LI H X . Sub-domain adaptation learning methodology[J]. Information Sciences, 2015(298): 237-256.
[5]	ZHU Y C , ZHUANG F Z , WANG J D ,et al. Deep subdomain adaptation network for image classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021,32(4): 1713-1722.
[6]	KRISHNA K , NARASIMHA M M . Genetic K-means algorithm[J]. IEEE Transactions on Systems,Man,and Cybernetics,Part B (Cybernetics), 1999,29(3): 433-439.
[7]	HARTIGAN J A , WONG M A . A K-means clustering algorithm[J]. Journal of the Royal Statistical Society Series C:Applied Statistics, 1979,28(1): 100-108.
[8]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.
[9]	HULL J J . A database for handwritten text recognition research[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994,16(5): 550-554.
[10]	GONG B Q , SHI Y , SHA F ,et al. Geodesic flow kernel for unsupervised domain adaptation[C]// Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2012: 2066-2073.
[11]	CHEN M , XU Z , WEINBERGER K ,et al. Marginalized denoising autoencoders for domain adaptation[EB]. 2012:arXiv:1206.4683.
[12]	JIANG S Y , XU Y H , SONG H J ,et al. Multi-instance transfer metric learning by weighted distribution and consistent maximum likelihood estimation[J]. Neurocomputing, 2018(321): 49-60.
[13]	ZUO H , ZHANG G Q , BEHBOOD V ,et al. Transfer learning in hierarchical feature spaces[C]// Proceedings of 2015 10th International Conference on Intelligent Systems and Knowledge Engineering (ISKE). Piscataway:IEEE Press, 2016: 183-188.
[14]	ZHAO Z , CHEN Y , LIU J ,et al. Cross-people mobile-phone based activity recognition[C]// Proceedings of the 22th International Joint Conference on Artificial Intelligence. Barcelona,Catalonia,Spain:AAAI Press, 2011: 2545-2550.
[15]	WEI Y , ZHU Y , LEUNG C ,et al. Instilling social to physical:Co-regularized heterogeneous transfer learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2016,30(1): 1338-1344.
[16]	MIHALKOVA L , HUYNH T , MOONEY R J . Mapping and revising Markov logic networks for transfer learning[C]// Proceedings of the 22nd National Conference on Artificial intelligence. Vancouver,BC,Canada:AAAI Press, 2007: 608-614.
[17]	MIHALKOVA L , MOONEY R J . Transfer learning by mapping with minimal target data[C]// Proceedings of the AAAI-08 Workshop on Transfer Learning for Complex Tasks. Chicago,IL,USA:AAAI Press, 2008: 1163-1168.
[18]	WANG M , DENG W . Deep visual domain adaptation:a survey[J]. Neurocomputing, 2018(312): 135-153.
[19]	LI J , HE R , YE H ,et al. Unsupervised domain adaptation of a pretrained cross-lingual language mode[J]. ArXiv, 2020,preprint arXiv:2011.11499.
[20]	DAI Y , LIU J , REN X C ,et al. Adversarial training based multi-source unsupervised domain adaptation for sentiment analysis[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(5): 7618-7625.
[21]	PAN S J , TSANG I W , KWOK J T ,et al. Domain adaptation via transfer component analysis[J]. IEEE Transactions on Neural Networks, 2011,22(2): 199-210.
[22]	LONG M S , WANG J M , DING G G ,et al. Transfer feature learning with joint distribution adaptation[C]// Proceedings of 2013 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2014: 2200-2207.
[23]	WANG J D , CHEN Y Q , HAO S J ,et al. Balanced distribution adaptation for transfer learning[C]// Proceedings of 2017 IEEE International Conference on Data Mining (ICDM). Piscataway:IEEE Press, 2017: 1129-1134.
[24]	COURTY N , FLAMARY R , TUIA D ,et al. Optimal transport for domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(9): 1853-1865.
[25]	DAS D , LEE C S G . Unsupervised domain adaptation using regularized hyper-graph matching[C]// Proceedings of 2018 25th IEEE International Conference on Image Processing (ICIP). Piscataway:IEEE Press, 2018: 3758-3762.
[26]	PEI Z Y , CAO Z J , LONG M S ,et al. Multi-adversarial domain adaptation[C]// Proceedings of the thirty-second AAAI Conference on Artificial Intelligence. New Orleans,LA,USA:AAAI Press, 2018.
[27]	KUMAR A , SATTIGERI P , WADHAWAN K ,et al. Co-regularized alignment for unsupervised domain adaptation-[EB]. 2018,arXiv,1811.05443.
[28]	ZHAO J C , DENG F , HE H B ,et al. Local domain adaptation for cross-domain activity recognition[J]. IEEE Transactions on Human-Machine Systems, 2021,51(1): 12-21.
[29]	ESTER M , KRIEGEL H P , SANDER J ,et al. A density-based algorithm for discovering clusters in large spatial databases with noise[C]// Proceedings of the Second International Conference on Knowledge Discovery and Data Mining. Portland,OR,USA:AAAI Press, 1996: 226-231.
[30]	徐晓, 丁世飞, 孙统风 ,等. 基于网格筛选的大规模密度峰值聚类算法[J]. 计算机研究与发展, 2018,55(11): 2419-2429.
	XU X , DING S F , SUN T F ,et al. Large-scale density peaks clustering algorithm based on grid screening[J]. Journal of Computer Research and Development, 2018,55(11): 2419-2429.
[31]	LONG M S , WANG J M , DING G G ,et al. Transfer joint matching for unsupervised domain adaptation[C]// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York:ACM, 2014: 1410-1417.
[32]	WANG J D , CHEN Y Q , YU H ,et al. Easy transfer learning by exploiting intra-domain structures[C]// Proceedings of 2019 IEEE International Conference on Multimedia and Expo (ICME). Piscataway:IEEE Press, 2019: 1210-1215.
[33]	ZHANG J , LI W Q , OGUNBONA P . Joint geometrical and statistical alignment for visual domain adaptation[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 5150-5158.
[34]	GONG B Q , GRAUMAN K , SHA F . Reshaping visual datasets for domain adaptation[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems-Volume 1. New York:ACM, 2013: 1286-1294.

Metrics

Recommended 0

No Suggested Reading articles found!

方法	BDA^[23]	OTDA^[24]	LDA^[28]	SCOAD（本文）
参数	μ	α , ,λη	λ, 1λ,η,k_t	K
子结构适配	不是	不是	是	是
子域内规划	不是	不是	不是	是
特征对齐	是	不是	不是	是

数据集	名称	特征维数	类别	样本个数
USPS+MNIST	USPS	256	10	1 800
	MNIST	256	10	2 000
Office-Caltech10	Caltech	800	10	1 123
	Webcam	800	10	295
	Amazon	800	10	958
	DSLR	800	10	157
Amazon Review	Kitchen	400	2	1 999
	DVDs	400	2	1 999
	Electronics	400	2	1 998
	Books	400	2	2 000

Substructure correlation adaptation transfer learning method based on K-means clustering

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 34

Related Articles 15

Metrics

Recommended 0

[1]	Zhimin HE, Yuzhe LIN, Yujie CHENG, Shi YAN. Downlink wireless resource allocation method of V2X based on wireless sensing assistance [J]. Telecommunications Science, 2022, 38(9): 60-70.
[2]	Wenjun ZHU, Sining WANG, Xiaoxin GAO, Qian ZHENG. Load forecasting based on knowledge flow and transfer learning [J]. Telecommunications Science, 2022, 38(5): 114-123.
[3]	Haibo ZHAO, Zhijun XIANG, Linsong XIAO. A big data framework for short-term power load forecasting using heterogenous data [J]. Telecommunications Science, 2022, 38(12): 103-111.
[4]	Ning JIN, Qingyang WANG. Research on clustering algorithm based 6G typical usage scenarios [J]. Telecommunications Science, 2022, 38(1): 121-131.
[5]	Xie LU, Lei XU, Manjun ZHANG. Safe hierarchical virtual network mapping method based on clustering [J]. Telecommunications Science, 2021, 37(9): 112-117.
[6]	Zihao LIU, Xiaojun JIA, Sulan ZHANG, Zhiling XU, Jun ZHANG. Vibe++ background segmentation method combining MeanShift clustering analysis and convolutional neural network [J]. Telecommunications Science, 2021, 37(3): 133-145.
[7]	Jihua WU, Pengyu ZHU, Zichen WU, Bin GU, Tao HONG, Bo GUO, Jing WANG, Jingyu WANG. Fault diagnosis and auto dispatchin of power communication network based on unsupervised clustering and frequent subgraph mining [J]. Telecommunications Science, 2021, 37(11): 51-63.
[8]	Xiang LI,Yuan LI,Zifei ZHANG,Zhe YANG. A density clustering-based network performance failure big data analysis algorithm [J]. Telecommunications Science, 2020, 36(9): 51-58.
[9]	Yajie LI,Yongli ZHAO,Shoudong LIU,Jie ZHANG. Overview of research on fiber nonlinear equalization algorithm based on artificial intelligence [J]. Telecommunications Science, 2020, 36(3): 61-70.
[10]	Yang SUN,Li SU,Xing ZHANG,Fengsheng WANG,Haitao DU. Method of short text strategy mining based on sub-semantic space [J]. Telecommunications Science, 2020, 36(3): 83-94.
[11]	Yingzhao ZHU. Review on heterogeneous transfer learning [J]. Telecommunications Science, 2020, 36(3): 100-110.
[12]	Dayan MA. Early warning prediction of external force destruction in transmission lines based on automatic clustering model [J]. Telecommunications Science, 2019, 35(3): 135-139.
[13]	Qizhu ZHONG, Xiuquan WU, Yaoman LUO. Research and application of new method based on MDT intelligent analysis of LTE radio interference [J]. Telecommunications Science, 2019, 35(10): 130-136.
[14]	Zeheng YUAN,Runlan TIAN,Xiaofeng WANG. Pre-sorting method of complex system radar signals [J]. Telecommunications Science, 2018, 34(9): 97-104.
[15]	Can LIU,Junna SHANG,Ruijiang LI,Keqiang YUE. Indoor dynamic environment lo calization algorithm based on transfer learning [J]. Telecommunications Science, 2018, 34(8): 98-108.