Energy Storage Arbitrage in Grid-Connected Micro-Grids Under Real-Time Market Price Uncertainty: A Double-Q Learning Approach

Abstract

Energy storage plays a significant role in improving the stability of distributed energy, improving power quality and peak regulation in the micro-grid system, which is of great significance to the sustainable development of energy. In grid-connected mode, energy storage is mainly used to reduce the operating costs of micro-grid. Real-time price arbitrage is an important source of energy storage revenue. It is feasible to design arbitrage strategies using Q-learning algorithm. Due to the overestimation of the Q learning algorithm, this paper proposes an arbitrage strategy method based on Double-Q learning. Compared with Q-learning algorithm, Double-Q learning can avoid overestimation and provide more stable and accurate arbitrage strategy for energy storage systems. Since the source of arbitrage in previous studies was limited to electricity prices alone, this paper considers joint arbitrage of electricity and carbon prices. The simulation results show that if adding fluctuate carbon prices to arbitrage sources, the arbitrage profits will increase by more than 110%.