2020 · 14 citations · 23 references
ComputingCloud ComputingEnergy ManagementMachine LearningOptimizationDeep LearningEnergy-efficient Cloud ComputingPower ControlPower ManagementPower OptimizationReinforcement LearningDeep Reinforcement LearningMinimizationNetwork Energy OptimizationPower AllocationPower ConsumptionPower Consumption ModelingPower Consumption OptimizationQ-learning
Cloud radio access network (CRAN) facilitates resource allocation (RA) by isolating remote radio heads (RRHs) from baseband units (BBUs). Traditional RA algorithms save energy by dynamically turning on/off RRHs and allocating power in each time slot. However, when the energy switching cost is considered, the decisions of turning on/off RRHs in adjacent time slots are correlated, which cannot be solved directly. Fortunately, deep reinforcement learning (DRL) can effectively model such problem, which motivates us to minimize the total power consumption subject to the constraints on per-RRH transmit power and user rates. Our starting point is the deep Q network (DQN), which is a combination of a neural network and Q-learning. In each time slot, DQN turns on /off a RRH yielding the largest Q-value (known as action value) prior to solving a power minimization problem for active RRHs. However, DQN yields Q-value overestimation issue, which stems from using the same network to choose the best action and to compute the target Qvalue of taking that action at the next state. To further increase the CRAN power savings, we propose a Double DQN-based framework by decoupling the action selection from the target Q-value generation. Simulation results indicate that the Double DQN-based RA method outperforms the DQN-based RA algorithm in terms of total power consumption.
23
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Deep Reinforcement Learning with Double Q-Learning
Hado van Hasselt, Arthur Guez, David Silver · 2016 · 6K citations · Full text
Applications of second-order cone programming
Miguel Sousa Lobo, Lieven Vandenberghe, Stephen Boyd et al. · Linear Algebra and its Applications · 1998 · 2.4K citations