IEEE Transactions on Signal Processing · 2012 · 41 citations · 26 references
Natural SciencesComputer SystemsMathematicsDistributed SystemsMarkov Decision ProcessesOptimizationDistributed ControlOptimal ControlPartially Observable Markov Decision ProcessesPower ControlStochastic ControlStochastic OptimizationAlmost Sure ConvergenceDecentralized LearningDecentralized Policy LearningDecentralized PomdpsDelay MinimizationDelay OptimizationPartially Observable Stochastic GamesStochastic Optimal Control
In this paper, we consider delay minimization for interference networks with renewable energy source, where the transmission power of a node comes from both the conventional utility power (ac power) and the renewable energy source. We assume the transmission power of each node is a function of the local channel state, local data queue state and local energy queue state only. We consider two delay optimization formulations, namely the decentralized partially observable Markov decision process (DEC-POMDP) and noncooperative partially observable stochastic game (POSG). In DEC-POMDP formulation, we derive a decentralized online learning algorithm to determine the control actions and Lagrangian multipliers (LMs) simultaneously, based on the policy gradient approach. Under some mild technical conditions, the proposed decentralized policy gradient algorithm converges almost surely to a local optimal solution. In the noncooperative POSG formulation, the transmitter nodes are noncooperative. We extend the decentralized policy gradient solution and establish the technical proof for almost-sure convergence of the learning algorithms. In both cases, the solutions are very robust to model variations.
26
Power management in energy harvesting sensor networks
Aman Kansal, Jason C. Hsu, Sadaf Zahedi et al. · ACM Transactions on Embedded Computing Systems · 2007 · 1.5K citations