Federated Ensembles: a literature review

Abstract

Abstract Federated learning (FL) allows machine learning algorithms to be applied to decentralized data when data sharing is not an option due to privacy concerns. Ensemble-based learning works by training multiple (weak) classifiers whose output is aggregated. Federated ensembles are ensembles applied to a federated setting, where each classifier in the ensemble is trained on one data location. The aim of this review is to provide an overview of the published literature on federated ensembles, their applications, the methods used, the challenges faced, the proposed solutions and their comparative performance. We searched for publications on federated ensembles on five databases (ACM Digital Library, IEEE, arXiv, Google scholar and Scopus) published after 2016. We found 26 articles describing studies either proposing federated ensemble applications or comparing federated ensembles to other federated learning approaches. Federated ensembles were used for a wide varied applications beyond classification. Advocates of federated ensemble mentioned their ability to handle local biases in data. In comparison to federated learning approaches, federated ensembles underperformed in small sample sizes and highly class imbalanced settings. Only 10 articles discussed privacy guarantees or additional privacy preserving techniques. Federated ensembles represent an interesting alternative to federated averaging algorithms that is inherently privacy preserving. They have proved their versatility but remain underutilized.