Substation Operation Sequence Inference Model Based on Deep Reinforcement Learning

Abstract

At present, substation operation ticket system is developed based on an expert system, which has some problems such as knowledge base redundancy, intelligence deficiency and automatic learning ability. To solve this problem, this paper proposes an operation sequence reasoning model based on the knowledge base of the Neo4j knowledge graph and DuelingDQN (Dueling Deep Q Network) algorithm. Firstly, the diagram structure model of substation main wiring was established using the Neo4j knowledge graph. Based on the diagram structure model, the operable equipment set of the operation task was searched to form the task space, action space and action selection model of DuelingDQN. The reward and punishment function was designed based on the “five defense” rules and the state change of equipment. Make DuelingDQN model and Neo4j model interact in real time, and automatically learn the operation sequence. The test results show that the method proposed in this paper can automatically deduce the correct operation steps under different wiring modes and realize the transfer within the station, which is of great significance to the intellectualization of the operation ticket system.