IEEE Journal on Selected Areas in Communications · 2019 · 142 citations · 32 references
ComputingComputer ScienceComputer SystemsMarkov Decision ProcessesReal-time SystemsSchedulingQueueing TheoryReal-time Resource ManagementReal-time SchedulingResource ProvisioningResource SchedulingResource-constrained SchedulingDynamic Resource ProvisioningNetwork Resource AllocationNetwork SlicingOptimal AllocationOptimal Resource AllocationReal-time Resource AllocationSemi-markov Decision ProcessesSemi-markov Modelling
Effective network slicing requires an infrastructure/network provider to deal with the uncertain demands and real-time dynamics of the network resource requests. Another challenge is the combinatorial optimization of numerous resources, e.g., radio, computing, and storage. This paper develops an optimal and fast real-time resource slicing framework that maximizes the long-term return of the network provider while taking into account the uncertainty of resource demands from tenants. Specifically, we first propose a novel system model that enables the network provider to effectively slice various types of resources to different classes of users under separate virtual slices. We then capture the real-time arrival of slice requests by a semi-Markov decision process. To obtain the optimal resource allocation policy under the dynamics of slicing requests, e.g., uncertain service time and resource demands, a Q-learning algorithm is often adopted in the literature. However, such an algorithm is notorious for its slow convergence, especially for problems with large state/action spaces. This makes Q-learning practically inapplicable to our case, in which multiple resources are simultaneously optimized. To tackle it, we propose a novel network slicing approach with an advanced deep learning architecture, called deep dueling, that attains the optimal average reward much faster than the conventional Q-learning algorithm. This property is especially desirable to cope with the real-time resource requests and the dynamic demands of the users. Extensive simulations show that the proposed framework yields up to 40% higher long-term average return while being few thousand times faster, compared with the state-of-the-art network slicing approaches.
32
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Martı́n Abadi, Ashish Agarwal, Paul Barham et al. · arXiv (Cornell University) · 2016 · 9.8K citations · Full text
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 9K citations · Full text
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2016 · 6.8K citations · Full text
Computing, Computer Science, Physics +17