Formal Assessment of Agreement and Similarity between an Open-Source and a Reference Industrial Device with an Application to a Low-Cost pH Logger

Abstract

Open-source devices are nowadays used in a vast number of research fields like medicine, education, agriculture, and sports, among others. In this work, an open-source, portable, low-cost pH logger, appropriate for in situ measurements, was designed and developed to assist in experiments on agricultural produce manufacturing. Τhe device was calibrated manually using pH buffers for values of 4.01 and 7.01. Then, it was tested by manually measuring the pH from the juice of citrus fruits. A waterproof temperature sensor was added to the device for temperature compensation when measuring the pH. A formal method comparison process between the open-source device and a Hanna HI9024 Waterproof pH Meter was designed to assess their agreement. We derived indices of agreement and graphical assessment tools using mixed-effects models. The advantages and disadvantages of interpreting agreement through the proposed procedure are discussed. In our illustration, the indices reported mediocre agreement and the subsequent similarity analysis revealed a fixed bias of 0.22 pH units. After recalibration, agreement between the devices improved to excellent levels. The process can be followed in general to avoid misleading or over-simplistic results of studies reporting solely correlation coefficients for formal comparison purposes.