Deep Q-Network-Based Cooperative Transmission Joint Strategy Optimization Algorithm for Energy Harvesting-Powered Underwater Acoustic Sensor Networks

Abstract

Cooperative transmission is a promising technology for underwater acoustic sensor networks (UASNs) to ensure the effective collection of underwater information. In this paper, we study the joint relay selection and power allocation problem to maximize the cumulative quality of information transmission in energy harvesting-powered UASNs (EH-UASNs). First, we formulate the process of cooperative transmission with joint strategy optimization as a Markov decision process model. In the proposed model, an effective state expression is presented to better reveal interactive relationship between learning and environment, thereby improving the learning ability. Then, we further propose a novel reward function which can guide nodes to adjust power strategy adaptively to balance instantaneous capacity and long-term quality of service (QoS) under the dynamic unpredictable energy harvesting. More specifically, we propose a deep Q-network-based resource allocation algorithm for EH-UASNs to solve the complex coupled strategy optimization problem without any prior underwater environment information. Finally, simulation results verify the superior performance of the proposed algorithm in improving the cumulative network capacity and reducing outages.