Affiliation:
1. College of Systems Engineering, National University of Defense Technology, Changsha 410073, China
Abstract
Federated Learning (FL) is a promising distributed machine learning framework that emphasizes privacy protection. However, inconsistencies between local optimization objectives and the global objective, commonly referred to as client drift, primarily arise due to non-independently and identically distributed (Non-IID) data, multiple local training steps, and partial client participation in training. The majority of current research tackling this challenge is mainly based on the empirical risk minimization (ERM) principle, while giving little consideration to the connection between the global loss landscape and generalization capability. This study proposes FedGAM, an innovative FL algorithm that incorporates Gradient Norm-Aware Minimization (GAM) to efficiently search for a local flat landscape. FedGAM specifically modifies the client model training objective to simultaneously minimize the loss value and first-order flatness, thereby seeking flat minima. To directly smooth the global flatness, we propose the more significant FedGAM-CV, which employs control variables to correct local updates, guiding each client to train models in a globally flat direction. Experiments on three datasets (CIFAR-10, MNIST, and FashionMNIST) demonstrate that our proposed algorithms outperform existing FL baselines, effectively finding flat minima and addressing the client drift problem.
Funder
General Program of the National Natural Science Foundation of China
Reference50 articles.
1. A comparative study of deep learning and Internet of Things for precision agriculture;Saranya;Eng. Appl. Artif. Intell.,2023
2. Subhashini, R., and Khang, A. (2023). The role of Internet of Things (IoT) in smart city framework. Smart Cities, CRC Press.
3. Aminizadeh, S., Heidari, A., Toumaj, S., Darbandi, M., Navimipour, N.J., Rezaei, M., Talebi, S., Azad, P., and Unal, M. (2023). The applications of machine learning techniques in medical data processing based on distributed computing and the Internet of Things. Comput. Methods Programs Biomed., 241.
4. Internet of Things and smart sensors in agriculture: Scopes and challenges;Rajak;J. Agric. Food Res.,2023
5. A review of AI teaching and learning from 2000 to 2020;Ng;Educ. Inf. Technol.,2023