Caught in the data quality trap: A case study from the evaluation of a new digital technology supporting routine health data collection in Southern Tanzania

Abstract

AbstractBackgroundHealth service data from Health Management Information Systems is important for decision-making at all health system levels. Data quality issues in low-and-middle-income countries hamper data use however.Smart Paper Technology, a novel digital-hybrid technology, was designed to overcome quality challenges through automated digitization. Here we assessed the impact of the novel system on data quality dimensions, metrics and indicators as proposed by the World Health Organization’sData Quality Review Toolkit.MethodsThis cross-sectional study was conducted between November 2019 and October 2020 in 13 health facilities sampled from 33 facilities of one district in rural Tanzania, where we implementedSmart Paper Technology. We assessed the technology’s data quality for maternal health care against the standardDistrict Health Information System-2applied in Tanzania.ResultsSmart Paper Technologyperformed slightly better than theDistrict Health Information System-2regardingconsistency between related indicatorsandoutliers. We found <10% difference between related indicators for 62% of the facilities for the new system versus 38% for the standard system in the reference year.Smart Paper Technologywas inferior toDistrict Health Information System-2data in terms ofcompleteness. We observed that data on 1stantenatal care visitswere complete ⍰ 90% in only 76% of facilities for the new system against 92% for the standard system. For the indicatorinternal consistency over time73%, 59% and 45% of client numbers for antenatal, labour and postnatal care recorded in the standard system were documented in the new system.Smart Paper Technologyforms were submitted in 83% of the months for all service areas.ConclusionOur results suggest that not all client encounters were documented inSmart Paper Technology, affecting data completeness and partly consistency. The novel system was unable to leverage opportunities from automated processes because primary documentation was poor. Low buy-in of policymakers and lack of internal quality assurance may have affected data quality of the new system. We emphasize the importance of including policymakers in evaluation planning to co-design a data quality monitoring system and to agree on a realistic way to ensure reporting of routine health data to national level.