Multimodal anomaly detection for high-speed train control system based on attention mechanism

Abstract

Abstract Accurate and rapid anomaly detection of train control systems is an inevitable requirement for ensuring the safe and efficient operation of high-speed railways. Currently, the manual offline fault diagnosis has issues such as ineffectiveness in fault locating and a relatively large scope of fault impact. In response, an anomaly detection model based on multimodal learning with the attention mechanism is proposed. According to the interrelated relationship between text logs and visual images representing equipment working status, a language-vision fusion two-stream multimodal neural network learning architecture is designed. The entire network structure, centred on the attention mechanism, learns the mapping relationship between inputs and outputs, simultaneously processes log generation of multiple sub-equipment, and separately focuses on the context of the text and changes in indicator light display at specific positions on the visual images. At the final decision-making layer, the learning results of language and vision are organically fused through logical operations, producing a unified output indicating the anomalous state of the system. Experimental results on real train operation datasets demonstrate the model’s superior performance in terms of precision and recall compared to other methods, validating its effectiveness.