Error Rates of Data Processing Methods in Clinical Research: A Systematic Review and Meta-Analysis

Abstract

Abstract Background: Over the last 30 years, empirical assessments of data accuracy in clinical research have been reported in the literature. Although there have been articles summarizing results reported in multiple papers, there has been little synthesis of these results. Further, although notable exceptions exist, little evidence has been obtained regarding the relative accuracy of different data processing methods. Methods: A systematic review of the literature was performed to identify clinical research studies that evaluated the quality of data obtained from data processing methods typically used in clinical research (e.g., medical record abstraction, optical scanning, single-data entry, and double-data entry). A total of 93 papers meeting our inclusion criteria were categorized according to their data processing methods. Quantitative information on data accuracy was abstracted from the articles and pooled. Meta-analysis of single proportions based on an inverse variance method and generalized linear mixed model approach of studies from the literature were used to derive an overall estimate of error rates across data processing methods for comparison. Results: Review of the literature indicated that the accuracy associated with data processing methods varies widely, with error rates ranging from 2 errors per 10,000 fields to 2,784errors per 10,000 fields. The medical record abstraction process for data acquisition in clinical research was associated with both high and highly variable error rates, with a variability of 3 orders of magnitude in accuracy (70 – 2,784 errors per 10,000 fields). Error rates for data processed with optical methods were comparable to data processed using single-data entry (2 – 358 vs. 4 – 650 per 10,000 fields, respectively). In comparison, double-data entry was associated with the lowest error rates (4 – 33 per 10,000 fields). Conclusions: Data processing and cleaning methods may explain a significant amount of the variability in data accuracy.