New Zealand’s Integrated Data Infrastructure (IDI): Value to date and future opportunities

Abstract

ABSTRACT ObjectivesStatistics New Zealand, the official statistics agency of the New Zealand (NZ) Government, has developed an Integrated Data Infrastructure (IDI). We aimed to review the value of the IDI to date from a research perspective and to consider the opportunities it provides. ApproachWe considered recent projects in which we have been involved: the New Zealand Census-Mortality and CancerTrends studies (NZCMS/CT), a Suicide Mortality Review Committee (SuMRC) feasibility trial, and a study on cardiovascular disease costs in NZ. We also considered IDI-related work published by other agencies and the views expressed at IDI meetings involving health and social sector personnel. ResultsSince 1993 in NZ, it has been possible to link all official health datasets using the National Health Index (NHI) number. However it was much more difficult to link health data with data from other sources. Studies often involved much ad hoc data linkage, with different versions of the same data being stored in multiple sites resulting in data integrity and security issues. The IDI provides one place to securely link the various data sources then make the anonymised data available for research for public good. The IDI provides the following advantages: Administrative data sources are available in one place nationally rather than many data silos. Health data can be linked with other administrative data eg, tax, education, crime, migration, welfare, housing, population census.  It facilitates collaboration between agencies and the sharing of code and associated documentation. Increased use of these data means there is more feedback to the source government agencies around data collection standards and data quality issues.  It allows for nationally-standardised processes for ensuring that only responsible and capable researchers with appropriate research questions have access to the relevant anonymised data. Perhaps the greatest long-term research potential is with the repeated measures data, allowing analyses with greater potential causal inference than previously available. All the above reduce the cost of doing research as well as assisting government departments with evaluation and costing studies. ConclusionsNZ is making good progress with integration of administrative data from the health and social sectors. The IDI is a valuable tool for advancing research that provides government departments and academic researchers with new insights into complex social issues such as crime and vulnerable children. It allows government to target limited resources to those areas where the greatest long-term benefit might be made.