A Dynamic Drilling Sampling Method and Evaluation Model for Big Streaming Data

Abstract

The big data sampling method for real-time and high-speed streaming data is prone to lose the value and information of a large amount of discrete data, and it is not easy to make an efficient and accurate evaluation of the value characteristics of streaming data. The SDSLA sampling method based on mineral drilling exploration can evaluate the valuable information of streaming data containing many discrete data in real-time, but when the range of discrete data is irregular, it has low sampling accuracy for discrete data. Based on the SDSLA algorithm, we propose a dynamic drilling sampling method SDDS, which takes well as the analysis unit, dynamically changes the size and position of the well, and accurately locates the position and range of discrete data. A new model SDVEM is further proposed for data valuation, which evaluates the sample set from discrete, centralized, and overall dimensions. Experiments show that compared with the SDSLA algorithm, the sample sampled by the SDDS algorithm has higher evaluation accuracy, and the probability distribution of the sample is closer to the original streaming data, with the AOCV indicator being nearly 10% higher. In addition, the SDDS algorithm can achieve over 90% accuracy, recall, and F1 score for training and testing neural networks with small sampling rates, all of which are higher than the SDSLA algorithm. In summary, the SDDS algorithm not only accurately evaluates the value characteristics of streaming data but also facilitates the training of neural network models, which has important research significance in big data estimation.