Algorithms to Reduce the Data File Size and Improve the Write Rate for Storing Sensor Reading Values in Hard Disk Drives for Measurements with Exceptionally High Sampling Rates

Abstract

This study aimed to enhance the data write performance in measurements with exceptionally high sampling rates, such as acoustic emission measurements. This is particularly crucial when employing conventional hard disk drives to store data. This study introduced algorithms for handling binary formats, thereby reducing the data file size, increasing write rates, and ultimately shortening data write times during measurements. The suggested approaches included utilizing specialized binary formats and implementing self-created buffers. These approaches resulted in a remarkable write time reduction of up to 40×. Furthermore, employing multiple drives for writing significantly enhanced performance compared with that of using a single drive. Therefore, the proposed algorithms offer promising results for managing large amounts of data in real time.