UAV-Aided Anti-Jamming Maritime Communications: A Deep Reinforcement Learning Approach

Abstract

Maritime communication suffers from reactive jammer that keeps silent when the channel is idle and launches jamming signals as soon as sensing the activity on the channel to disrupt the maritime communication and even cause the maritime tracking and monitoring task failure. In this paper, we propose a deep reinforcement learning based maritime anti-jamming communication scheme that applies the unmanned aerial vehicle (UAV) as the relay to forward the message between ships. This scheme enables both the transmitting ship and the UAV to select the power control policy without relying on the jamming model and the message generation model. Instead, the policies are chosen based on the current state consisting of the distances, the received signal strength indication, the received jamming power and the bit error rate (BER). We design two dueling neutral networks respectively to select the power control policy and update the network weights to reduce the overestimation. The dueling architecture consists of two streams that estimate the value of the current state and the state-dependent advantage function of the power control policy to accelerate the learning speed. A UAV-aided anti-jamming maritime communication game is formulated and the performance bound including the BER of marine messages and the energy consumption of the transmitting ship and the UAV is derived based on the Stackelberg equilibrium. Simulation results show that the proposed scheme reduces the BER of the marine messages, and saves the energy consumption of the transmitting ship and the UAV to avoid being detected by the reactive jammer.