Abstract:

With the continuous growth and advancement of the Internet and information technology, continuous growth and advancement of the Internet and information technology.Nevertheless, these applications’ vulnerabilities pose a severe threat to information security and users’ privacy.Fuzzing was widely used as one of the main tools for automatic vulnerability detection due to its ease of vulnerability recurrence and low false positive errors.It generates test cases randomly and executes the application by optimization in terms of coverage or sample generation to detect deeper program paths.However, the mutation operation in fuzzing is blind and tends to make the generated test cases execute the same program path.Consequently, traditional fuzzing tests have problems such as low efficiency, high randomness of inputs generation and limited pertinence of the program structure.To address these problems, a directional fuzzing based on deep reinforcement learning was proposed, which used deep reinforcement learning networks with information obtained by staking program to guide the selection of the inputs.Besides, it enabled fast approximation and inspection of the program paths that may exist vulnerabilities.The experimental results showed that the proposed approach had better performance than the popular fuzzing tools such as AFL and AFLGO in terms of vulnerability detection and recurrence on the LAVA-M dataset and real applications like LibPNG and Binutils.Therefore, the approach can provide support for further vulnerability mining and security research.

Key words: vulnerability mining, fuzzing test, deep reinforcement learning, program path