WGM-dSAGA: Federated Learning Strategies with Byzantine Robustness Based on Weighted Geometric Median

Abstract

Federated learning techniques accomplish federated modeling and share global models without sharing data. Federated learning offers a good answer to complex data and privacy security issues. Although there are many ways to target federated learning, Byzantine attacks are the ones we concentrate on. Byzantine attacks primarily impede learning by tampering with the local model parameters provided by a client to the master node throughout the federation learning process, leading to a final global model that diverges from the optimal solution. To address this problem, we combine aggregation rules with Byzantine robustness using a gradient descent optimization algorithm based on variance reduction. We propose a WGM-dSAGA method with Byzantine robustness, called weighted geometric median-based distributed SAGA. We replace the original mean aggregation strategy in the distributed SAGA with a robust aggregation rule based on weighted geometric median. When less than half of the clients experience Byzantine attacks, the experimental results demonstrate that our proposed WGM-dSAGA approach is highly robust to different Byzantine attacks. Our proposed WGM-dSAGA algorithm provides the optimal gap and variance under a Byzantine attack scenario.