基于符号执行和N-scope复杂度的代码混淆度量方法

doi:10.11959/j.issn.2096-109x.2022085

摘要/Abstract

摘要：

代码混淆可有效对抗逆向工程等各类 MATE 攻击威胁，作为攻击缓和性质的内生安全技术发展较为成熟，对代码混淆效果的合理度量具有重要价值。代码混淆度量研究相对较少，针对代码混淆弹性的度量方法与泛化性、实用性度量方法相对缺乏。符号执行技术广泛应用于反混淆攻击，其生成遍历程序完整路径输入测试集的难度可为混淆弹性度量提供参考，然而基于程序嵌套结构的对抗技术可显著降低符号执行效率，增加其混淆弹性参考误差。针对上述问题，提出结合符号执行技术和N-scope复杂度的代码混淆度量方法，该方法首先基于程序符号执行时间定义程序混淆弹性；其次提出适配符号执行的N-scope复杂度，定义程序混淆强度同时增强符号执行对多层嵌套结构程序的混淆弹性度量鲁棒性；进而提出结合动态分析与静态分析的混淆效果关联性分析，通过对程序进行符号执行与控制流图提取量化混淆效果。面向 C 程序构建了该度量方法的一种实现框架并验证，实验对3个公开程序集及其混淆后程序集约4 000个程序进行混淆效果度量，度量结果表明，提出的度量方法在较好地刻画混淆效果的同时拥有一定的泛化能力与实用价值；模拟真实混淆应用场景给出了该度量方法的使用样例，为混淆技术使用人员提供有效的混淆技术度量与技术配置参考。

关键词: 代码混淆, 混淆度量, 符号执行, N-scope

Abstract:

Code obfuscation has been well developed as mitigated endogenous security technology, to effectively resist MATE attacks (e.g.reverse engineering).And it also has important value for the reasonable metrics of code obfuscation effect.Since symbolic execution is widely used in anti-obfuscation attacks, metrics for code obfuscation resilience can refer to the efforts of generating input test set for executing all program paths.However, some adversarial techniques could reduce the symbol execution efficiency significantly based on the nested structure of the program and increase the error of the resilience reference.To solve the above problems, a metrics for code obfuscation was proposed based on symbolic execution and N-scope complexity.The obfuscation resilience was defined with symbolic execution time and obfuscation potency was defined based on the proposed N-scope complexity for better robustness in measuring the resilience of multi-nested structure programs.Furthermore, the correlation analysis of obfuscation effect was proposed and the effect was quantified by symbolic execution and control flow diagram extraction of programs.Over 4000 obfuscated programs from 3 open-sourced assemblies were evaluated with proposed metrics in the experiment, which indicated the generalization performance and practicality of the metrics.And an example of this metrics application was presented in a simulated obfuscation scenario which provided references of obfuscation technology metrics and obfuscation configuration for obfuscation users.

Key words: code obfuscation, obfuscation metrics, symbolic execution, N-scope

中图分类号:

TP393

肖玉强, 郭云飞, 王亚文. 基于符号执行和N-scope复杂度的代码混淆度量方法[J]. 网络与信息安全学报, 2022, 8(6): 123-134.

Yuqiang XIAO, Yunfei GUO, Yawen WANG. Metrics for code obfuscation based on symbolic execution and N-scope complexity[J]. Chinese Journal of Network and Information Security, 2022, 8(6): 123-134.

图/表 12

图1

图2

表1

表2

图3

图4

图5

图6

图7

图8

表3

表4

参考文献 30

[1]	AKHUNZADA A , SOOKHAK M , ANUAR N ,et al. Man-at-the-end attacks:analysis,taxonomy,human aspects,motivation and future directions[J]. Journal of Network and Computer Applications, 2015,48(6): 44-57.
[2]	VAN DEN BROECK J , COPPENS B , DE SUTTER B . Flexible software protection[J]. Computers ＆ Security, 2022,10(2): 6-36.
[3]	耿普, 祝跃飞 . 一种基于分支条件混淆的代码加密技术[J]. 计算机研究与发展, 2019,56(10): 2183-2192.
	GENG P , ZHU Y F . A code encrypt technique based on branch condition obfuscation[J]. Journal of Computer Research and Development, 2019,56(10): 2183-2192.
[4]	陈喆, 王志, 王晓初 ,等. 基于代码移动的二进制程序控制流混淆方法[J]. 计算机研究与发展, 2015,52(8): 1902-1909.
	CHEN Z , WANG Z , WANG XC ,et al. Using code mobility to obfuscate control flow in binary codes[J]. Journal of Computer Research and Development, 2015,52(8): 1902-1909.
[5]	AHIRE P , ABRAHAM J . Mechanisms for source code obfuscation in C:novel techniques and implementation[C]// 2020 International Conference on Emerging Smart Computing and Informatics. 2020.
[6]	PIZZOLOTTO D , FELLIN R , CECCATO M . OBLIVE:seamless code obfuscation for Java programs and Android Apps[C]// 2019 IEEE 26th International Conference on Software Analysis,Evolution and Reengineering. 2019.
[7]	VAN DEN BROECK J , COPPENS B , DE SUTTER B . Obfuscated integration of software protections[J]. International Journal of Information Security, 2021,20(1): 73-101.
[8]	CECCATO M , ANDREA C , PAOLO F ,et al. A large study on the effect of code obfuscation on the quality of Java code[J]. Empirical Software Engineering, 2015,20(6): 1486-1524.
[9]	LARSEN P , HOMESCU A , BRUNTHALER S ,et al. SoK:automated software diversity[C]// Symposium on Security and Privacy. 2014.
[10]	XU H , ZHOU Y , MING J ,et al. Layered obfuscation:a taxonomy of software obfuscation techniques for layered security[J]. Cybersecurity, 2020,3(1): 1-18.
[11]	PENG Y , CHEN Y , SHEN B . An adaptive approach to recommending obfuscation rules for java bytecode obfuscators[C]// 2019 IEEE 43rd Annual Computer Software and Applications Conference. 2019.
[12]	COLLBERG C , THOMBORSON C , LOW D . A taxonomy of obfuscating transformations[R]. 1997.
[13]	ANCKAERT B , MADOU M , DE SUTTER B ,et al. Program obfuscation:quantitative approach[C]// ACM Workshop on Quality of Protection, 2007.
[14]	KIRK SR , JENKINS S . Information theory-based software metrics and obfuscation[J]. Journal of Systems and Software, 2004,72(2): 179-186.
[15]	MOHSEN R , PINTO AM . Evaluating obfuscation security:A quantitative approach[C]// International Symposium on Foundations and Practice of Security. 2016.
[16]	MOHSEN R . Quantitative measures for code obfuscation security[D]. London:Imperial College London, 2016.
[17]	RODRíGUEZ-VELIZ M , NU?EZ-MUSA Y , LIMA R S . Call graph obfuscation and diversification:an approach[J]. IET Inf Secur, 2020,14(2): 241-252.
[18]	TSAI HY , HUANG YL , WAGNER D . A graph approach to quantitative analysis of control-flow obfuscating transformations[J]. IEEE Transactions on Information Forensics and Security, 2009,4(2): 257-267.
[19]	GAO D , REITER MK , SONG D . BinHunt:automatically finding semantic differences in binary programs[C]// International Conference on Information and Communications Security. 2008.
[20]	LUO L , MING J , WU D ,et al. Semantics-based obfuscation-resilient binary code similarity comparison with applications to software plagiarism detection[C]// ACM Sigsoft International Symposium on Foundations of Software Engineering. 2014.
[21]	PUCELLA R , SCHNEIDER FB . Independence from obfuscation:a semantic framework for diversity[J]. Computers ＆ Security, 2010,18(5): 701-749.
[22]	DALLA M , GIACOBAZZI R . Semantic-based code obfuscation by abstract interpretation[C]// International Colloquium on Automata,Languages,and Programming. 2005.
[23]	VITICCHIE A , REGANO L , TORCHIANO M ,et al. Assessment of source code obfuscation techniques[C]// International Working Conference on Source Code Analysis and Manipulation. 2016.
[24]	CECCATO M , DI PENTA M , NAGRA J ,et al. The effectiveness of source code obfuscation:an experimental assessment[C]// International Conference on Program Comprehension. 2009.
[25]	BANESCU S , COLLBERG C , GANESH V ,et al. Code obfuscation against symbolic execution attacks[C]// Annual Conference on Computer Security Applications. 2016.
[26]	MAJUMDAR A , DRAPE S , THOMBORSON C . Metrics-based evaluation of slicing obfuscations[C]// International Symposium on Information Assurance and Security. 2007.
[27]	MOLNAR D , LI XC , WAGNER DA . Dynamic test generation to find integer bugs in x86 binary linux programs[C]// USENIX Security Symposium. 2009.
[28]	WANG Z , MING J , JIA C , GAO D . Linear obfuscation to combat symbolic execution[C]// European Symposium on Research in Computer Security. 2011.
[29]	ZUSE H . Software complexity:measures and methods[R]. Berlin,Walter de Gruyter GmbH ＆ Co KG, 1991.
[30]	KARNICK M , MACBRIDE J , MCGINNIS S ,et al. A qualitative analysis of Java obfuscation[C]// IASTED International Conference on Software Engineering and Applications. 2006.

程序符号	均值	标准差
if×1	0.03	0
if×2	0.04	0.007
if嵌套if	0.04	0.005
loop×1	0.06	0.05
loop×2	0.21	0.292
loop+if	0.068	0.021
loop嵌套if	0.52	0.50
loop嵌套loop	4.31	9.57
loop输入无关	0.03	/
loop输入有关	0.12	/

混淆技术	模块名称	描述
模糊谓词	AddO4/16	在控制流中添加模糊谓词，谓词数量为4或16
函数拆分	Split	将函数拆分为小片段并进行外联
控制流平坦	Flatten	将顺序执行的程序结构转换为多分支循环结构
代码虚拟化	Virtualize	将函数转换为拥有唯一字节码的解释器进行执行
数学运算加密	Encode Arithmetic	将整数运算转换为相同语义下更复杂的表达式

混淆技术	模块名称	描述
函数复制	Copy	复制一个函数
冗余清扫	CleanUp	删除未使用变量，非必要函数名等
谓词更新	UpdateOpaques	添加更新模糊谓词的代码
虚假控制流	ImplicitFlow	在基本块间构造冗余控制流
反别名分析	AntiAliasAnalysis	通过用间接函数调用替换直接函数调用来转换代码
反污点分析	AntiTaintAnalysis	结合虚假控制流对抗动态污点分析工具
熵增益	Generate Entropy	增加程序运行时的随机性
伪信息泄露	Leak	插入伪装成泄露秘密函数的代码

混淆技术	N-scope复杂度	符号执行时间/min
混淆技术	N-scope复杂度	均值	中位数	标准差
1.AddO16-Copy-Flatten-ImpF-Leak	0.57	3.95	2.76	3.40
2.EncA-Virt-ImpF-Leak-AntiTA-CleanUP	0.46	3.60	2.37	3.18
3.EncA-UpdateO-Virt-GenrEntr-EncL	0.88	3.33	1.71	3.05
4.EncA-UpdateO-GenrEntr-AntiAA-EncL	0.21	3.33	2.88	2.62
5.EncA-ImpF-AntiTA-EncL-CleanUP	0.21	3.27	2.99	2.52
6.EncA-UpdateO-Virt-AntiAA-Split	0.86	3.12	1.59	2.84
7.EncA-Virt-GenrEntr-Leak-AntiAA	0.85	2.99	1.51	2.85
8.Flat-GenrEntr-ImpF-EncL-CleanUP	0.51	2.96	1.58	2.55
9.Flat-UpdateO-ImpF-AntiTA-Split-EncL	0.46	2.80	1.55	2.51
10.ImpF -Flat-AntiTA-Split- CleanUp	0.43	2.59	1.26	2.45
11.Virt-GenrEntr-ImpF-EncL-CleanUp	0.83	2.56	1.22	2.28
12.Copy-UpdateO-Virt-Split-CleanUP	0.82	2.54	1.17	2.39
13.AddO16-GenrEntr-ImpF-EncL-CleanUP	0.22	2.48	1.26	2.21
14.EncA-ImpF-AntiAA-Split-EncL	0.20	2.47	1.34	2.10
15.AddO16-GenrEntr-Leak-EncL-CleanUP	0.22	2.47	1.40	2.07
16.AddO16-UpdateO-Virt-GenrEntr-ImpF	0.85	2.43	1.13	2.19
17.Flat-UpdateO-Virt-ImpF-Leak-AntiTA	0.84	2.37	1.08	2.21
18.UpdateO-GenrEntr-ImpF-AntiTA-Split	0.21	2.05	0.92	1.91
19.Copy-Flat-UpdateO-EncL-CleanUp	0.52	0.52	0.16	0.50