Intelligent Anti-Jamming Decision Algorithm of Bivariate Frequency Hopping Pattern Based on DQN With PER and Pareto

Abstract

To improve the anti-jamming performance of frequency hopping system in complex electromagnetic environment, a Deep Q-Network algorithm with priority experience replay (PER) based on Pareto samples (PPER-DQN) is proposed, which makes intelligent decision for bivariate FH pattern. The system model, state-action space and reward function are designed based on the main parameters of the FH pattern. The DQN is used to improve the flexibility of the FH pattern. Based on the definition of Pareto dominance, the PER based on the TD-error and immediate reward is proposed. To ensure the diversity of the training set, it is formed by Pareto sample set and several random samples. When selecting Pareto sample, the confidence coefficient is introduced to modify its priority. It guarantees the learning value of the training set and improves the learning efficiency of DQN. The simulation results show that the efficiency, convergence speed and stability of the algorithm are effectively improved. And the generated bivariate FH pattern has better performance than the conventional FH pattern.