Applied Sciences · 2021 · 69 citations · 15 references
ComputingComputer ScienceCybersecurityArtificial IntelligenceMarkov Decision ProcessesPenetration TestingDeep LearningExploration-exploitation TradeoffPolicy LearningReinforcement LearningReinforcement Learning FoundationsAction Space FormulationDeep Reinforcement LearningExperience ReplayExplorationExploration In Reinforcement LearningLearning From Sparse RewardsQ-learningSparse Reward Learning
Penetration testing is an effective way to test and evaluate cybersecurity by simulating a cyberattack. However, the traditional methods deeply rely on domain expert knowledge, which requires prohibitive labor and time costs. Autonomous penetration testing is a more efficient and intelligent way to solve this problem. In this paper, we model penetration testing as a Markov decision process problem and use reinforcement learning technology for autonomous penetration testing in large scale networks. We propose an improved deep Q-network (DQN) named NDSPI-DQN to address the sparse reward problem and large action space problem in large-scale scenarios. First, we reasonably integrate five extensions to DQN, including noisy nets, soft Q-learning, dueling architectures, prioritized experience replay, and intrinsic curiosity model to improve the exploration efficiency. Second, we decouple the action and split the estimators of the neural network to calculate two elements of action separately, so as to decrease the action space. Finally, the performance of algorithms is investigated in a range of scenarios. The experiment results demonstrate that our methods have better convergence and scaling performance.
15
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Mastering the game of Go with deep neural networks and tree search
David Silver, Aja Huang, Chris J. Maddison et al. · Nature · 2016 · 15.8K citations
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Oriol Vinyals, I. Babuschkin, Wojciech Marian Czarnecki et al. · Nature · 2019 · 3.4K citations
Richard Bellman · Indiana University Mathematics Journal · 1957 · 2.4K citations
Dueling Network Architectures for Deep Reinforcement Learning
Ziyu Wang, Tom Schaul, Matteo Hessel et al. · arXiv (Cornell University) · 2015 · 1.8K citations · Full text