Testing federated analytics across secure data environments using differing statistical approaches on cross-disciplinary data

Abstract

ABSTRACTBACKGROUNDIntroducing data-driven technologies into health systems can enhance population health and streamline care delivery. The use of diverse and geographically varied data is key for tackling health and societal challenges, despite associated technical, ethical, and governance complexities. This study explored the efficacy of federated analytics using general linear models (GLMs) and machine learning (ML) models, comparing outcomes with non-federated data analysis.METHODSA Conditional Transformation Generative Adversarial Network was used to create two synthetic datasets (training set: N=10,000; test set: N=1,000), using real-world data from 381 asthma patients. To simulate a federated environment, the resulting data were distributed across nodes in a Microsoft Azure Trusted Research Environment (TRE). GLMs (one-way ANOVA) and ML models (gradient boosted decision trees) where then produced, using both federated and non-federated approaches. The consistency of predictions produced by the ML models were then compared between approaches, with predictive accuracy of the models quantified by the area under the receiver operating characteristic curve (AUROC).FINDINGSGLMs produced from federated data distributed between two TREs were identical to those produced using a non-federated approach. However, ML models produced by federated and non-federated approaches, and using different data distributions between TREs, were non-identical. Despite this, when applied to the test set, the classifications made by the federated models were consistent with the non-federated model in 84.7-90.4% of cases, which was similar to the consistency of repeated non-federated models (90.9-91.5%). Consequently, overall predictive accuracies for federated and non-federated models were similar (AUROC: 0.663-0.669).INTERPRETATIONThis study confirmed the robustness of GLMs utilising ANOVA within a federated framework, yielding consistent outcomes. Moreover, federated ML models demonstrated a high degree of classification agreement, with comparable accuracy to traditional non-federated models. These results highlight the viability of federated approaches for reliable and accurate data analysis in sensitive domains.