Indirect Customer-to-Customer Energy Trading With Reinforcement Learning

Abstract

In this paper, we explore the role of emerging energy brokers (middlemen) in a localized event-driven market (LEM) at the distribution level for facilitating indirect customer-to-customer energy trading. This proposed LEM does not aim to replace any existing energy service or become the best market model; but instead to diversify the energy ecosystem at the edge of distribution networks. In light of this philosophy, the market mechanism will provide additional options for customers and prosumers who have the willingness to directly participate in the retail electricity market occasionally, on top of using existing utility services. It also helps in improving market efficiency and encouraging local-level power balance, while taking into account the characteristics of customers’ behavior. The energy trading process will be built as a Markov decision process with some reinforcement learning and data-driven methods applied. Some economic concepts, like <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">search friction</i> , related to this kind of typical search cost involved market model are also discussed.