2019 · 61 citations · 21 references
ComputingComputer ScienceArtificial IntelligenceFormal MethodsMachine LearningCongestion ControlDeductive VerificationDeep LearningExplainable Machine LearningModel CheckingReinforcement LearningDeep Reinforcement LearningFlow SchedulingFormal VerificationInternet Congestion ControlNetwork VerificationPacket ClassificationSecurity Property VerificationVerification Of Neural Network Controllers
Deep reinforcement learning (RL) has recently been successfully applied to networking contexts including routing, flow scheduling, congestion control, packet classification, cloud resource management, and video streaming. Deep-RL-driven systems automate decision making, and have been shown to outperform state-of-the-art handcrafted systems in important domains. However, the (typical) non-explainability of decisions induced by the deep learning machinery employed by these systems renders reasoning about crucial system properties, including correctness and security, extremely difficult. We show that despite the obscurity of decision making in these contexts, verifying that deep-RL-driven systems adhere to desired, designer-specified behavior, is achievable. To this end, we initiate the study of formal verification of deep RL and present Verily, a system for verifying deep-RL-based systems that leverages recent advances in verification of deep neural networks. We employ Verily to verify recently-introduced deep-RL-driven systems for adaptive video streaming, cloud resource management, and Internet congestion control. Our results expose scenarios in which deep-RL-driven decision making yields undesirable behavior. We discuss guidelines for building deep-RL-driven systems that are both safer and easier to verify.
21
Mastering the game of Go with deep neural networks and tree search
David Silver, Aja Huang, Chris J. Maddison et al. · Nature · 2016 · 15.8K citations
Mastering the game of Go without human knowledge
David Silver, Julian Schrittwieser, Karen Simonyan et al. · Nature · 2017 · 9.2K citations
Engineering, Educational Psychology, Cognitive Psychology +7
Neural Adaptive Video Streaming with Pensieve
Hongzi Mao, Ravi Netravali, Mohammad Alizadeh · 2017 · 1.3K citations · Full text