异构迁移学习研究综述

doi:10.11959/j.issn.1000-0801.2020060

摘要/Abstract

摘要：

异构迁移学习突破了同构迁移学习要求源域和目标域特征空间必须相同的界限，实现对异构数据的分析挖掘和知识迁移，进一步促进数据复用，为机器学习领域开拓更大的应用范围。首先介绍了迁移学习的定义与分类，然后深入阐述异构迁移学习的研究现状，并对其应用场景进行分析，最后指出了异构迁移学习当前存在的问题及未来可能的研究方向。

关键词: 异构迁移学习, 源域, 目标域, 知识迁移, 数据复用

Abstract:

Heterogeneous transfer learning breaks through the boundary that the feature space of source domain and target domain must be the same.It realizes analysis mining and knowledge migration of heterogeneous data.It further promotes data reuse and opens up a wider range of applications for the field of machine learning.Firstly,the definition and classification of transfer learning was introduced.Then,the research status of heterogeneous transfer learning was elaborated and its application scenarios were analyzed.At last,the existing problems and the possible research direction in the future were pointed out.

Key words: heterogeneous transfer learning, source domain, target domain, knowledge transfer, data reuse

中图分类号:

TP181

朱应钊. 异构迁移学习研究综述[J]. 电信科学, 2020, 36(3): 100-110.

Yingzhao ZHU. Review on heterogeneous transfer learning[J]. Telecommunications Science, 2020, 36(3): 100-110.

图/表 9

图1

表1

表2

表3

图2

图3

图4

图5

表4

参考文献 57

[1]	王志宏, 杨震 . 人工智能技术研究及未来智能化信息服务体系的思考[J]. 电信科学, 2017,33(5): 1-11.
	WANG Z H , YANG Z . Research on artificial intelligence technology and the future intelligent information service architecture[J]. Telecommunications Science, 2017,33(5): 1-11.
[2]	闫玉光 . 面向异构特征空间的迁移学习研究[D]. 广州:华南理工大学, 2019.
	YAN Y G . Research on transfer learning for heterogeneous feature space[D]. Guangzhou:South China University of Technology, 2019.
[3]	DAY O , KHOSHGOFTAAR T M . A survey on heterogeneous transfer learning[J]. Journal of Big Data, 2017,4(1).
[4]	李茂莹, 杨柳, 胡清华 . 同构迁移学习理论和算法研究进展[J]. 南京信息工程大学学报（自然科学版）, 2019(3): 269-277.
	LI M Y , YANG L , HU Q H . A survey on theories and algorithms about homogeneous transfer learning[J]. Journal of Nanjing University of Information Science and Technology (Natural Science Edition), 2019,11(3): 269-277.
[5]	刘建伟, 孙正康, 罗雄麟 . 域自适应学习研究进展[J]. 自动化学报, 2014,40(8): 1576-1600.
	LIU J W , SUN Z K , LUO X L . Review and research development on domain adaptation learning[J]. Acta Automatica Sinica, 2014,40(8): 1576-1600.
[6]	刘鑫鹏, 栾悉道, 谢毓湘 ,等. 迁移学习研究和算法综述[J]. 长沙大学学报, 2018,32(5): 33-36,41.
	LIU X P , LUAN X D , XIE Y X ,et al. Transfer learning research and algorithm review[J]. Journal of Changsha University, 2018,32(5): 33-36,41.
[7]	王惠 . 迁移学习研究综述[J]. 电脑知识与技术, 2017,13(32): 209-211.
	WANG H . Transfer learning research[J]. Computer Knowledge and Technology, 2017,13(32): 209-211.
[8]	DAI W , YANG Q , XUE G R . Boosting for transfer learning[C]// Proceedings of the 24th International Conference on Machine Learning. New York:ACM Press, 2007: 193-200.
[9]	LONG M , CAO Y , WANG J . Learning transferable features with deep adaptation networks[C]// Proceedings of the 32nd International Conference on Machine Learning. New York:ACM Press, 2015: 97-105.
[10]	YOSINSKI J , CLUNE J , BENGIO Y . How transferable are features in deep neural networks[C]// Proceedings of Annual Conference on Neural Information Processing Systems. Cambridge:MIT Press, 2014: 3320-3328.
[11]	DAVIS J , DOMINGOS P . Deep transfer via second-order Markov logic[C]// Proceedings of the 26th International Conference on Machine Learning. New York:ACM Press, 2009: 217-224.
[12]	PAN S J , YANG Q . A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010,22(10): 1345-1359.
[13]	WEISS K , KHOSHGOFTAAR T M , WANG D D . A survey of transfer learning[J]. Journal of Big Data, 2016,3(1): 1-40.
[14]	WANG Z , DAI Z , POCZOS B ,et al. Characterizing and avoiding negative transfer[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019： 11293-11302.
[15]	YAO Y , DORETTO G . Boosting for transfer learning with multiple sources[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2010: 1855-1862.
[16]	SHA F , SHI Y , GONG B ,et al. Geodesic flow kernel for unsupervised domain adaptation[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2012: 2066-2073.
[17]	CHANG S F , XU D , DUAN L . Exploiting web images for event recognition in consumer videos:A multiple source domain adaptation approach[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2012: 1338-1345.
[18]	TOMMASI T , ORABONA F , CAPUTO B . Safety in numbers:learning categories from few examples with multi model knowledge transfer[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2010: 3081-3088.
[19]	徐嵚嵛 . 基于迁移学习的表情识别算法研究[D]. 南京:南京邮电大学, 2017.
	XU Q Y . Facial expression recognition based on transfer learning algorithm[D]. Nanjing:Nanjing University of Posts and Telecommunications, 2017.
[20]	TAN B , ZHANG Y , PAN S J ,et al. Distant domain transfer learning[C]// Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2017: 2604-2610.
[21]	BLITZER J , MCDONALD R , PEREIRA F . Domain adaptation with structural correspondence learning[C]// Proceedings of Conference on Empirical Methods in Natural Language Processing.[S.l.:s.n]. 2006: 120-128.
[22]	PRETTENHOFER P , STEIN B . Cross-language text classification using structural correspondence learning[C]// Proceedings of Annual Meeting of the Association for Computational Linguistics.[S.l.:s.n]. 2010: 1118-1127.
[23]	SHI X , LIU Q , FAN W ,et al. Transfer learning on heterogenous feature spaces via spectral transformation[C]// Proceedings of the 10th IEEE International Conference on Data Mining. Piscataway:IEEE Press, 2010: 1049-1054.
[24]	WANG C , MAHADEVAN S . Heterogeneous domain adaptation using manifold alignment[C]// Proceedings of International Joint Conference on Artificial Intelligence. San Francisco:Morgan Kaufmann Press, 2011: 1541-1546.
[25]	ZHU Y , CHEN Y , LU Z ,et al. Heterogeneous transfer learning for image classification[C]// Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2011: 1-6.
[26]	QI G J , AGGARWAL C , HUANG T . Towards semantic knowledge propagation from text corpus to web images[C]// Proceedings of 20th International Conference on World Wide Web. New York:ACM Press, 2011: 297-306.
[27]	DUAN L , XU D , TSANG I W . Learning with augmented features for heterogeneous domain adaptation[C]// Proceedings of the 29th International Conference on Machine Learning. New York:ACM Press, 2012: 711-718.
[28]	VAPNIK V , . Principles of risk minimization for learning theory[C]// Proceedings of Annual Conference on Neural Information Processing Systems. Cambridge:MIT Press, 1991: 831-838.
[29]	LI W , DUAN L , XU D ,et al. Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014,36(6): 1134-1148.
[30]	KULIS B , SAENKO K , DARRELL T . What you saw is not what you get:domain adaptation using asymmetric kernel transforms[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2011: 1785-1792.
[31]	HAREL M , MANNOR S . Learning from multiple outlooks[C]// Proceedings of the 28th International Conference on Machine Learning. New York:ACM Press, 2011: 401-408.
[32]	ZHOU J T , TSANG I W . Heterogeneous domain adaptation for multiple classes[C]// Proceedings of International conference on Artificial Intelligence and Statistics.[S.l.:s.n]. 2014: 1095-1103.
[33]	NAM J , Kim S . Heterogeneous defect prediction[C]// Proceedings of the 10th Joint Meeting on Foundations of Software Engineering. New York:ACM Press, 2015: 508-519.
[34]	ZHOU J T , PAN S , TSANG I W ,et al. Hybrid heterogeneous transfer learning through deep learning[C]// Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2017: 2213-2220.
[35]	NIR F , DAN G , MOISES G . Bayesian network classifiers[J]. Machine Learning, 1997,29(2-3): 131-163.
[36]	JOACHIMS T , . Text categorization with support vector machines:learning with many relevant features[C]// Proceedings of the 10th European Conference on Machine Learning. Berlin:Springer Press, 1998: 137-142.
[37]	QUINLAN J R . Induction of decision trees[J]. Machine Learning, 1986,1(1): 81-106.
[38]	SHIMODAIRA H . Improving predictive inference under covariate shift by weighting the log-likelihood function[J]. Journal of Statistical Planning and Inference, 2000,90(2): 227-244.
[39]	杨柳 . 融合文本信息的图像分类和标注关键问题研究[D]. 北京:北京交通大学, 2016.
	YANG L . Research on key issues of images classification and annotation by fusing text information[D]. Beijing:Beijing Jiaotong University, 2016.
[40]	DAI W , CHEN Y , XUE G R ,et al. Translated learning:transfer learning across different feature spaces[C]// Proceedings of Annual Conference on Neural Information Processing Systems. Cambridge:MIT Press, 2008: 353-360.
[41]	NIEBLES J C , CHEN C W , LI F F . Modeling temporal structure of decomposable motion segments for activity classification[C]// Proceedings of the 11th European Conference on Computer Vision. Berlin:Springer Press, 2010: 392-405.
[42]	谭瑶, 饶文碧 . 异构复合迁移学习的视频内容标注方法[J]. 计算机应用, 2018,38(6): 1547-1553.
	TAN Y , RAO W B . Heterogeneous compound transfer learning method for video content annotation[J]. Journal of Computer Applications, 2018,38(6): 1547-1553.
[43]	LORENZO B , MATTIA M . Domain adaptation problems:a DASVM classification technique and a circular validation strategy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010,32(5): 770-787.
[44]	倪超, 陈翔, 刘望舒 ,等. 基于特征迁移和实例迁移的跨项目缺陷预测方法[J]. 软件学报, 2019,30(5): 1308-1329.
	NI C , CHEN X , LIU W S ,et al. Cross-project defect prediction method based on feature transfer and instance transfer[J]. Journal of Software, 2019,30(5): 1308-1329.
[45]	刘树毅 . 基于特征迁移的跨项目软件缺陷预测[D]. 呼和浩特:内蒙古师范大学, 2019.
	LIU S Y . Cross-project software defect prediction based on feature transfer[D]. Hohhot:Inner Mongolia Normal University, 2019.
[46]	蒋兰华 . 基于迁移学习的软件缺陷预测技术研究[J]. 微型电脑应用, 2019,35(4): 93-94,99.
	JIANG L H . Research on software defect prediction technology based on transfer learning[J]. Microcomputer Applications, 2019,35(4): 93-94,99.
[47]	吴琦 . 基于迁移学习的跨项目软件缺陷预测[D]. 长春:吉林大学, 2018.
	WU Q . Cross-project defect prediction based on transfer learning[D]. Changchun:Jilin University, 2018.
[48]	MA Y , LUO G , ZENG X ,et al. Transfer learning for cross-company software defect prediction[J]. Information ＆Software Technology, 2012,54(3): 248-256.
[49]	TURHAN B , MENZIES T , BENER A B ,et al. On the relative value of cross-company and within-company data for defect prediction[J]. Empirical Software Engineering, 2009,14(5): 540-578.
[50]	NAM J , PAN S J , KIM S . Transfer defect learning[C]// Proceedings of the 35th International Conference on Software Engineering. Piscataway:IEEE Press, 2013: 382-391.
[51]	JING X , WU F , DONG X ,et al. Heterogeneous cross-company defect prediction by unified metric representation and CCA-based transfer learning[C]// Proceedings of the 10th Joint Meeting on Foundations of Software Engineering. New York:ACM Press, 2015: 496-507.
[52]	谭琦 . 面向跨领域文档分类的异构迁移学习算法研究[D]. 广州:华南理工大学, 2014.
	TAN Q . Research on heterogeneous machine learning for cross-domain document classification[D]. Guangzhou:South China University of Technology, 2014.
[53]	汪来富, 金华敏, 刘东鑫 ,等. 面向网络大数据的安全分析技术应用[J]. 电信科学, 2017,33(3): 112-118.
	WANG L F , JIN H M , LIU D X ,et al. Application of security analysis technology for network big data[J]. Telecommunications Science, 2017,33(3): 112-118.
[54]	吴涛, 刘韬, 王斌 . 安徽联通企业级大数据平台构建及应用实践[J]. 电信科学, 2018,34(1): 135-147.
	WU T , LIU T , WANG B . Construction and application of Anhui Unicom enterprise big data platform[J]. Telecommunications Science, 2018,34(1): 135-147.
[55]	MAGDA F , MARCEL J . Asymmetric heterogenous transfer learning:a survey[C]// Proceedings of the 6th International Conference on Data Science,Technology and Applications. New York:ACM Press, 2017: 17-27.
[56]	蹇松雷, 卢凯 . 复杂异构数据的表征学习综述[Z]. 2020.
	JIAN S L , LU K . Survey on representation learning of complex heterogeneous data[Z]. 2020.
[57]	JIAN S , HU L , CAO L ,et al. Metric-based auto-instructor for learning mixed data representation[C]// Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2018: 3318-3325.

迁移学习方式	源域与目标域特征空间	需解决的问题
同构迁移学习	相同	域自适应学习
异构迁移学习	不同	特征空间对齐+域自适应学习

迁移学习方式		源域/目标域标注情况
归纳式迁移学习	多任务学习	有/少量
	自学习	没有/少量
直推式迁移学习	—	有/没有
无监督迁移学习	—	没有/没有

迁移学习方式	特点
基于实例的迁移学习	赋予源域实例一定的权重对其进行复用
基于特征的迁移学习	基于特征变换减少源域与目标域之间的差距
基于网络的迁移学习	找出源域与目标域网络模型之间的共享参数
基于关系的迁移学习	挖掘不同领域之间的关系相似性

方法	迁移类别	源数据	目标数据	多源	通用解决方案	负迁移
CLSCL	基于对称特征	标记	未标记	不支持	不支持	无
HeMap	基于对称特征	标记	部分标记	不支持	支持	无
DAMA	基于对称特征	标记	部分标记	支持	支持	无
HTLIC	基于对称特征	未标记	大量标记	不支持	支持	无
TTI	基于对称特征	标记	部分标记	不支持	不支持	无
HFA	基于对称特征	标记	部分标记	不支持	支持	无
SHFA	基于对称特征	标记	部分标记	不支持	支持	无
ARC-t	基于非对称特征	标记	部分标记	不支持	支持	无
MOMAP	基于非对称特征	标记	部分标记	支持	支持	无
SHFR	基于非对称特征	标记	部分标记	不支持	支持	无
HDP	基于非对称特征	标记	未标记	不支持	支持	无
HHTL	基于非对称特征	标记	未标记	不支持	不支持	无