Analysis of Industrial Sensor Data Using Statistical and Regression Methods

Abstract

Today's industrial landscape is primarily driven by rapid and effective data processing and evaluation. Consequently, industries should devote considerable attention and resources towards real-time examination of the large data sets acquired, enabling timely extraction of vital information for outlier detection, fake data identification, and predictive analysis to mitigate unforeseen expenses. This rigorous process of data analysis necessitates the employment of a diverse set of algorithms that align with the specific objectives, spanning a wide spectrum of potential solutions. In this manuscript, we demonstrate how Apache Spark's unified engine can be harnessed for conducting statistical analysis of time series data, thereby expediting industrial data analysis processes. Furthermore, we examine and implement both linear and random forest regression models within the context of the demonstrated use case.