Research on Outlier Detection Methods for Dam Monitoring Data Based on Post-Data Classification

Abstract

Safety monitoring of hydraulic structures is a critical task in the field of hydraulic engineering construction. This study developed a method for preprocessing and classifying monitoring data for the identification of gross errors in hydraulic structures. By utilizing linear regression and wavelet analysis techniques, it effectively differentiated various waveform characteristics in data sets, such as Sinusoidal Wave Cyclical, Triangular Wave Cyclical, Seasonal Cyclical, and Weakly Cyclical growth types. In the experiments for gross error identification, the 3σ algorithm, K-medoids algorithm, and Isolation Forest algorithm were applied to test the data. The results showed that the K-medoids algorithm excelled in processing Sinusoidal Wave Cyclical Data Sets; the 3σ algorithm adapted better to Triangular Wave Cyclical Data Sets; the Isolation Forest algorithm performed well in handling data sets with significant anomalies or atypical fluctuations and excelled in scenarios with strong seasonality and large data fluctuations; and for complex Weakly Cyclical Growth Data Sets, all three algorithms were less effective, indicating the potential need for more advanced analysis methods or a combination of multiple techniques. Testing on actual engineering data further confirmed the importance of using specific gross error identification techniques for special data types after data set pre-classification, providing a more effective technical solution for the safety monitoring of hydraulic structures.