Policy Selection and Scheduling of Cyber-Physical Systems with Denial-of-Service Attacks via Reinforcement Learning

Abstract

This paper focuses on policy selection and scheduling of sensors and attackers in cyber-physical systems (CPSs) with multiple sensors under denial-of-service (DoS) attacks. DoS attacks have caused enormous disruption to the regular operation of CPSs, and it is necessary to assess this damage. The state estimation of the CPSs plays a vital role in providing real-time information about their operational status and ensuring accurate prediction and assessment of their security. For a multi-sensor CPS, this paper is different from utilizing robust control methods to characterize the state of the system against DoS attacks, but rather positively analyzes the optimal policy selection of the sensors and the attackers through dynamic programming ideology. To optimize the strategies of both sides, game theory is employed as a means to study the dynamic interaction that occurs between the sensors and the attackers. During the policy iterative optimization process, the sensors and attackers dynamically learn and adjust strategies by incorporating reinforcement learning. In order to explore more state information, the restriction on the set of states is relaxed, i.e., the transfer of states is not limited compulsorily. Meanwhile, the complexity of the proposed algorithm is decreased by introducing a penalty in the reward function. Finally, simulation results show that the proposed algorithm can effectively optimize policy selection and scheduling for CPSs with multiple sensors.