IEEE Communications Letters · 2018 · 11 citations · 9 references
ManagementComputer SystemsEnergy ManagementEnergy SystemsMarkov Decision ProcessesEnergy EfficiencyEnergy Management SystemsReinforcement LearningReinforcement Learning FoundationsEnergy Harvesting SystemsEnergy-delay TradeoffEnergy-delay Tradeoff OptimizationEnergy-efficient CommunicationFinite-state Markov ChainsMarkov Decision ModelsState AbstractionState-space Abstraction
Efficient energy management methods are expected to become cornerstones for realizing the energy-harvesting-based device-to-device communication underlaying cellular network (EH-DCCN). But, the quality of service (such as delay performance) also needs to be considered when we pursue the high energy efficiency. Our aim is to investigate a tradeoff perspective between energy efficiency and delay under the EH-DCCN. Thus, we model a discrete-time and finite-state Markov decision process to study a long-term energy efficiency and delay tradeoff (EDT) problem. Simultaneously, an experience-based irrelative state-action abstraction (EISA) algorithm based on Q-learning (QL) is proposed to address the EDT problem and accelerate the convergence rate. Simulation results reveal that the EISA algorithm can achieve the same optimal performance as the typical QL method with rapid convergence speed.
9