机器学习安全及隐私保护研究进展

doi:10.11959/j.issn.2096-109x.2018067

Abstract

Abstract:

As an important method to implement artificial intelligence,machine learning technology is widely used in data mining,computer vision,natural language processing and other fields.With the development of machine learning,it brings amount of security and privacy issues which are getting more and more attention.Firstly,the adversary model was described according to machine learning.Secondly,the common security threats in machine learning was summarized,such as poisoning attacks,adversarial attacks,oracle attacks,and major defense methods such as regularization,adversarial training,and defense distillation.Then,privacy issues such were summarized as stealing training data,reverse attacks,and membership tests,as well as privacy protection technologies such as differential privacy and homomorphic encryption.Finally,the urgent problems and development direction were given in this field.

Key words: machine learning, security threats, defense technology, privacy

CLC Number:

TP309.2

Lei SONG, Chunguang MA, Guanghan DUAN. Machine learning security and privacy:a survey[J]. Chinese Journal of Network and Information Security, 2018, 4(8): 1-11.

Figures/Tables 9

References 62

[1]	GHORBEL A , GHORBEL M , JMAIEL M . Privacy in cloud computing environments:a survey and research challenges[J]. Journal of Supercomputing, 2017,73(6): 2763-2800.
[2]	SILVER D , HUANG A , MADDISON C J ,et al. Mastering the game of go with deep neural networks and tree search[J]. Nature, 2016,529(7587): 484-489.
[3]	BARRENO M , NELSON B , SEARS R ,et al. Can machine learning be secure?[C]// ACM Symposium on Information,Computer and Communications Security. 2006: 16-25.
[4]	KEARNS M , LI M . Learning in the presence of malicious errors[J]. SIAM Journal on Computing, 1993,22(4): 807-837.
[5]	BIGGIO B , NELSON B , LASKOV P . Support vector machines under adversarial label noise[J]. Journal of Machine Learning Research, 2011,20(3): 97-112.
[6]	BIGGIO B , NELSON B , LASKOV P . Poisoning attacks against support vector machines[C]// International Coference on International Conference on Machine Learning. 2012: 1467-1474.
[7]	MEI S , ZHU X . Using machine teaching to identify optimal training-set attacks on machine learners[C]// AAAI. 2015: 2871-2877.
[8]	BIGGIO B , DIDACI L , FUMERA G ,et al. Poisoning attacks to compromise face templates[C]// International Conference on Biometrics. 2013: 1-7.
[9]	KLOFT M , LASKOV P . Security analysis of online anomaly detection[J]. Journal of Machine Learning Research, 2010,13(1): 3681-3724.
[10]	C SZEGEDY , W ZAREMBA , I SUTSKEVER , ,et al. Intriguing properties of neural networks[C]// 2014 International Conference on Learning Representations.Computational and Biological Learning Society. 2014.
[11]	PAPERNOT N , MC D P , SINHA A ,et al. Towards the science of security and privacy in machine learning[J]. arXiv preprint arXiv:1611.03814, 2016.
[12]	GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples[C]// International Conference on Learning Representations. 2015.
[13]	KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial machine learning at scale[J]. arXiv preprint arXiv:1611.01236, 2017.
[14]	DONG Y P , LIAO F Z , PANG T Y ,et al. Boosting adversarial attacks with momentum[J]. arXiv preprint arXiv:1710.06081, 2017.
[15]	MIYATO T , MAEDA S , KOYAMA M ,et al. Virtual adversarial training:a regularization method for supervised and semi-supervised learning[J]. arXiv preprint 1704.03976, 2017.
[16]	MOOSAVI-DEZFOOLI S , FAWZI A , FROSSARD P . DeepFool:a simple and accurate method to fool deep neural networks[C]// IEEE Conference on Computer Vision and Pattern Recognition. 2016: 2574-2582.
[17]	PAPERNOT N , MCDANIEL P , JHA S ,et al. The limitations of deep learning in adversarial settings[C]// IEEE European Symposium on Security and Privacy. 2016: 372-387.
[18]	SU J , VARGAS D V , KOUICHI S . One pixel attack for fooling deep neural networks[J]. arXiv preprint arXiv:1710.08864, 2017.
[19]	LOWD D , MEEK C . Adversarial learning[C]// The eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining. 2005: 641-647.
[20]	MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O ,et al. Universal adversarial perturbations[C]// IEEE Conference on Computer Vision and Pattern Recognition. 2017.
[21]	PAPERNOT N , MCDANIEL P , GOODFELLOW I ,et al. Practical black-box attacks against machine learning[C]// 2017 ACM on Asia Conf on Computer and Communications Security. 2017: 506-519.
[22]	PAPERNOT N , MCDANIEL P , GOODFELLOW I . Transferability in machine learning:from phenomena to black-box attacks using adversarial samples[J]. arXiv preprint arXiv:1605.07277, 2016.
[23]	GU S X , RIGAZIO L . Towards deep neural network architectures robust to adversarial examples[J]. arXiv preprint arXiv:1412.5068, 2014.
[24]	LYU C , HUANG K , LIANG H N . A unified gradient regularization family for adversarial examples[C]// IEEE International Conference on Data Mining. 2016: 301-309.
[25]	ZHAO Q Y , GRIFFIN L D . Suppressing the unusual:towards robust cnns using symmetric activation functions[J]. arXiv preprint arXiv:1603.05145, 2016.
[26]	ROZSA A , GUNTHER M , BOULT T E . Towards robust deep neural networks with BANG[J]. arXiv preprint arXiv:1612.00138, 2016.
[27]	GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples[C]// International Conference on Learning Representations. Computational and Biological Learning Society, 2015.
[28]	HUANG R , XU B , SCHUURMANS D ,et al. Learning with a strong adversary[J]. arXiv preprint arXiv:1511.03034, 2015.
[29]	TRAMèR F , KURAKIN A , PAPERNOT N ,et al. ensemble adversarial training:attacks and defenses[J]. arXiv preprint arXiv:1705.07204, 2017.
[30]	PAPERNOT N , MCDANIEL P , WU X ,et al. Distillation as a defense to adversarial perturbations against deep neural networks[C]// IEEE Symp on Security and Privacy. 2016: 582-597.
[31]	HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network[J]. arXiv preprint arXiv:1503.02531, 2015.
[32]	PAPERNOT N , MCDANIEL P . Extending defensive distillation[J]. arXiv preprint arXiv:1705.05264, 2017.
[33]	BULòS R , BIGGIO B , PILLAI I ,et al. Randomized prediction games for adversarial machine learning[J]. IEEE transactions on neural networks and learning systems, 2017,28(11): 2466-2478.
[34]	HARDT M , MEGIDDO N , PAPADIMITRIOU C ,et al. Strategic classification[C]// 2016 ACM conference on innovations in theoretical computer science. 2016: 111-122.
[35]	BRüCKNER M , KANZOW C , SCHEFFER T . Static prediction games for adversarial learning problems[J]. Journal of Machine Learning Research, 2012,13(Sep): 2617-2654.
[36]	METZEN J H , GENEWEIN T , FISCHER V ,et al. On detecting adversarial perturbations[J]. arXiv preprint arXiv:1702.04267, 2017.
[37]	LU JIAJUN , ISSARANON T , FORSYTH D . SAFETYNET:Detecting and rejecting adversarial examples robustly[J]. arXiv preprint arXiv:1704.00103, 2017.
[38]	HITAJ B , ATENIESE G ， PEREZ-CRUZ F . Deep models under the GAN:information leakage from collaborative deep learning[C]// ACM Sigsac Conference. 2017: 603-618.
[39]	FREDRIKSON M , LANTZ E , JHA S ,et al. Privacy in pharmacogenetics:an end-to-end case study of personalized warfarin dosing[C]// The 23rd Usenix Security Symposium. 2014: 17-32.
[40]	FREDRIKSON M , JHA S , RISTENPART T . Model inversion attacks that exploit confidence information and basic countermeasures[C]// The 22nd ACM Sigsac Conference on Computer and Communications Security. 2015: 1322-1333.
[41]	ATENIESE G , MANCINI L V , SPOGNARDI A ,et al. Hacking smart machines with smarter ones:How to extract meaningful data from machine learning classifiers[J]. International Journal of Security and Networks, 2015,10(3): 137-150.
[42]	SHOKRI R , STRONATI M , SONG C ,et al. Membership inference attacks against machine learning models[J]. arXiv preprint arXiv:1610.05820, 2016.
[43]	TRAMER F , ZHANG F , JUELS A ,et al. Stealing machine learning models via prediction apis[J]. arXiv preprint arXiv:1609.02943, 2016.
[44]	GENTRY , CRAIG , Fully homomorphic encryption using ideal lattices[J]. Stoc, 2009,9(4): 169-178.
[45]	DOWLIN N , RAN G B , LAINE K ,et al. CryptoNets:applying neural networks to encrypted data with high throughput and accuracy[C]// Radio and Wireless Symposium. 2016: 76-78.
[46]	HESAMIFARD E , TAKABI H , GHASEMI M ,et al. Privacy-preserving machine learning in cloud[C]// The 2017 on Cloud Computing Security Workshop. 2017: 39-43.
[47]	BARYALAI M , JANG-JACCARD J , LIU D . Towards privacy-preserving classification in neural networks[C]// IEEE Privacy,Security and Trust. 2017: 392-399.
[48]	XIE P , BILENKO M , FINLEY T ,et al. Crypto-nets:neural networks over encrypted data[J]. Computer Science, 2014.
[49]	STONE M H . The generalized weierstrass approximation theorem[J]. Mathematics Magazine, 1948,21(4): 167-184.
[50]	ZHANG Q , YANG L , CHEN Z . Privacy preserving deep computation model on cloud for big data feature learning[J]. IEEE Transactions on Computers, 2016,65(5): 1351-1362.
[51]	DWORK C , MCSHERRY F , NISSIM K ,et al. Calibrating noise to sensitivity in private data analysis[C]// The Third conference on Theory of Cryptography. 2006: 265-284.
[52]	ABADI M , CHU A , GOODFELLOW I ,et al. Deep learning with differential privacy[C]// 2016 ACM Sigsac Conference on Computer and Communications Security. 2016: 308-318.
[53]	PAPERNOT N , ABADI M , ERLINGSSON U ,et al. Semi- supervised knowledge transfer for deep learning from private training data[J]. arXiv preprint arXiv:1610.05755, 2016.
[54]	BEAULIEUJONES B K , WU Z S , WILLIAMS C J ,et al. Privacy-preserving generative deep neural networks support clinical data sharing[J]. bioRxiv, 2017.
[55]	郭鹏, 钟尚平, 陈开志 ,等. 差分隐私 GAN 梯度裁剪阈值的自适应选取方法[J]. 网络与信息安全学报, 2018,4(5): 10-20.
	GUO P , ZHONG S P , CHEN K Z ,et al. Adaptive selection method of differential privacy[J]. Chinese Journal of Network and Information Security, 2018,4(5): 10-20.
[56]	SHOKRI R , SHMATIKOV V . Privacy-preserving deep learning[C]// The 22nd ACM SIGSAC Conference on Computer and Communications Security. 2015: 1310-1321.
[57]	LIU M , JIANG H , CHEN J ,et al. A collaborative privacy-preserving deep learning system in distributed mobile environment[C]// International Conference on Computational Science and Computational Intelligence. 2017: 192-197.
[58]	LE T P , AONO Y , HAYASHI T ,et al. Privacy-preserving deep learning via additively homomorphic encryption[J]. IEEE Transactions on Information Forensics ＆ Security, 2018,13(5): 1333-1345.
[59]	MCMAHAN B , RAMAGE D . Federated learning:collaborative machine learning without centralized training data[J]. Google Research Blog, 2017.
[60]	BONAWITZ K , IVANOV V , KREUTER B ,et al. Practical secure aggregation for privacy-preserving machine learning[C]// 2017 ACM Sigsac Conference on Computer and Communications Security. 2017: 1175-1191.
[61]	MCMAHAN H B , MOORE E , RAMAGE D ,et al. Federated learning of deep networks using model averaging[J]. arXiv preprint arXiv:1502.01710v5, 2016.
[62]	OSSIA S A , SHAMSABADI A S , TAHERI A ,et al. A hybrid deep learning architecture for privacy-preserving mobile analytics[J]. arXiv preprint arXiv:1703.02952, 2017.

Metrics

Recommended 0

No Suggested Reading articles found!

阶段	敌手策略	敌手目标	敌手能力	敌手知识
训练阶段	投毒攻击	完整性/可用性	修改训练数据	有限知识
预测阶段	对抗攻击询问攻击	完整性/可用性完整性/可用性	制作对抗样本访问目标模型	黑盒/白盒黑盒

攻击类型	访问模型输出	获取模型内部结构	针对目标攻击	非针对目标攻击
			L-BFGS^[10]、	BIM^[13]、
白盒攻击	√	√	FGSM^[12]、	Deepfool^[16]
			JSMA^[17]
黑盒攻击	√			文献[20-22]

阶段	敌手策略	敌手目标	敌手能力	敌手知识
训练阶段	窃取训练数据	机密性	获得训练数据	有限知识
预测阶段	逆向攻击	机密性	提取模型/训练数据信息	黑盒/白盒
	成员推理攻击	机密性	访问目标模型	黑盒

隐私威胁	训练阶段	预测阶段	访问模型输出	访问模型内部结构	提取模型结构信息	提取训练数据信息
文献[38]	√		√	√		√
文献[41]		√	√	√		√
文献[43]		√	√		√
文献[39,40,42]		√	√			√

Machine learning security and privacy:a survey

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 62

Related Articles 15

Metrics

Recommended 0

[1]	Lijun ZU, Yalin CAO, Xiaohua MEN, Zhihui LYU, Jiawei YE, Hongyi LI, Liang ZHANG. Adaptive selection method of desensitization algorithm based on privacy risk assessment [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 49-59.
[2]	Saite CHEN, Weihai LI, Yuanzhi YAO, Nenghai YU. Location privacy protection method based on lightweight K-anonymity incremental nearest neighbor algorithm [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 60-72.
[3]	Ruiqi XIA, Manman LI, Shaozhen CHEN. Identification on the structures of block ciphers using machine learning [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 79-89.
[4]	Feng YU, Qingxin LIN, Hui LIN, Xiaoding WANG. Privacy-enhanced federated learning scheme based on generative adversarial networks [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 113-122.
[5]	Jinyin CHEN, Rongchang LI, Guohan HUANG, Tao LIU, Haibin ZHENG, Yao CHENG. Survey on vertical federated learning: algorithm, privacy and security [J]. Chinese Journal of Network and Information Security, 2023, 9(2): 1-20.
[6]	Min XIAO, Faying MAO, Yonghong HUANG, Yunfei CAO. Anonymous trust management scheme of VANET based on attribute signature [J]. Chinese Journal of Network and Information Security, 2023, 9(2): 33-45.
[7]	Jianlong XU, Jian LIN, Yusen LI, Zhi XIONG. Distributed user privacy preserving adjustable personalized QoS prediction model for cloud services [J]. Chinese Journal of Network and Information Security, 2023, 9(2): 70-80.
[8]	Zhe SUN, Hong NING, Lihua YIN, Binxing FANG. Preliminary study on the construction of a data privacy protection course based on a teaching-in-practice range [J]. Chinese Journal of Network and Information Security, 2023, 9(1): 178-188.
[9]	Fenghua LI, Hui LI, Ben NIU, Weidong QIU. Academic connotation and research trends of privacy computing [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 1-8.
[10]	Xue BAI, Baodong QIN, Rui GUO, Dong ZHENG. Two-party cooperative blind signature based on SM2 [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 39-51.
[11]	Min XIAO, Tao YAO, Yuanni LIU, Yonghong HUANG. Dynamic and efficient vehicular cloud management scheme with privacy protection [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 70-83.
[12]	Chenxin LU, Bing CHEN, Ning DING, Liquan CHEN, Ge WU. Identity-based anonymous cloud auditing scheme with compact tags [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 156-168.
[13]	Shengzhi MING, Jianming ZHU, Zhiyuan SUI, Xian ZHANG. Online medical privacy protection strategy under information value-added mechanism [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 169-177.
[14]	Zuobin YING, Yichen FANG, Yiwen ZHANG. Privacy-preserving federated learning framework with dynamic weight aggregation [J]. Chinese Journal of Network and Information Security, 2022, 8(5): 56-65.
[15]	Xian ZHANG, Jianming ZHU, Zhiyuan SUI, Shengzhi MING. Analysis on anonymity and regulation of digital currency transactions based on game theory [J]. Chinese Journal of Network and Information Security, 2022, 8(5): 150-157.