IEEE Transactions on Neural Networks and Learning Systems · 2019 · 210 citations · 50 references
EngineeringPower Grid OperationGame TheoryMultistage GameMulti-agent LearningNew SolutionTransmission LinesAttack SimulationScada SecurityStochastic GameSystems EngineeringMechanism DesignComputer EngineeringComputer ScienceSmart Grid SecurityGamesMarkov Decision ProcessReinforcement Learning SolutionSmart GridEnergy ManagementBusinessControl System Security
Existing smart‑grid security studies focus on attacker tactics and cascading failures, largely ignoring defender strategies, and most use one‑shot game theory rather than dynamic models. This work proposes a multistage (dynamic) attacker‑defender game solved via reinforcement learning to determine optimal attack sequences for objectives such as line outages or generation loss. The attacker learns a sequence of line‑attack actions while the defender protects selected lines, with cascading failure outcomes fed back after each step; the approach is tested on W&W 6‑bus and IEEE 39‑bus systems and compared to one‑shot attacks. Simulation results show the reinforcement‑learning framework can identify optimal attack sequences for multiple objectives and that the attacker’s learned information enables the defender to improve system security.
Existing smart grid security research investigates different attack techniques and cascading failures from the attackers' viewpoints, while the defenders' or the operators' protection strategies are somehow neglected. Game theoretic methods are applied for the attacker-defender games in the smart grid security area. Yet, most of the existing works only use the one-shot game and do not consider the dynamic process of the electric power grid. In this paper, we propose a new solution for a multistage game (also called a dynamic game) between the attacker and the defender based on reinforcement learning to identify the optimal attack sequences given certain objectives (e.g., transmission line outages or generation loss). Different from a one-shot game, the attacker here learns a sequence of attack actions applying for the transmission lines and the defender protects a set of selected lines. After each time step, the cascading failure will be measured, and the line outage (and/or generation loss) will be used as the feedback for the attacker to generate the next action. The performance is evaluated on W&W 6-bus and IEEE 39-bus systems. A comparison between a multistage attack and a one-shot attack is conducted to show the significance of the multistage attack. Furthermore, different protection strategies are evaluated in simulation, which shows that the proposed reinforcement learning solution can identify optimal attack sequences under several attack objectives. It also indicates that attacker's learned information helps the defender to enhance the security of the system.
50
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 8.9K citations · Full text
Catastrophic cascade of failures in interdependent networks
Sergey V. Buldyrev, Roni Parshani, Gerald Paul et al. · Nature · 2010 · 4.2K citations · Full text