A Novel Intelligent Anti-Jamming Algorithm Based on Deep Reinforcement Learning Assisted by Meta-Learning for Wireless Communication Systems

Abstract

In the field of intelligent anti-jamming, deep reinforcement learning algorithms are regarded as key technical means. However, the learning process of deep reinforcement learning algorithms requires a stable learning environment to ensure its effectiveness. Moreover, the inherent limitations of deep reinforcement learning algorithms mean that they can only demonstrate excellent learning capabilities on specific tasks with constant parameters. When parameters change, they can only resample and relearn to converge. In a changing jamming environment, its stability and convergence speed may be challenged, thereby affecting its robustness and generalization capabilities. Aiming at the naive yet unique similarity characteristics of the communication anti-jamming problem, this paper designs a new Meta-PPO deep reinforcement learning algorithm that combines Proximal Policy Optimization (PPO) and MAML meta-learning ideas. The proposed algorithm engrafts the principle of meta-learning used in the Model Agnostic Meta-Learning (MAML) model onto the Proximal Policy Optimization (PPO)-based schemes, enabling the communication systems to harness its prior learned experiences acquired from previous anti-jamming tasks to facilitate and speed up its optimal decision-making process when faced with incoming jamming attacks with similar features. The proposed algorithm is verified through computer simulation analyses and the results show that the proposed novel Meta-PPO algorithm can outperform traditional DQN- and PPO-based algorithms in terms of better robustness and generalization abilities, which can be used to enhance the anti-jamming capabilities of wireless communication systems.