Transferable face image privacy protection based on federated learning and ensemble models

Abstract

AbstractFace image features represent significant user privacy concerns. Face images cannot be privately transferred under existing privacy protection methods, and data across various social networks are unevenly distributed. This paper proposes a method for face image privacy protection based on federated learning and ensemble models. A federated learning model based on distributed data sets was established by means of federated learning. On the client side, a local facial recognition model was obtained by local face data training and used as the input of PcadvGAN to train PcadvGAN for several rounds. On the server side, a parameter aggregator based on a differential evolutionary algorithm was established as the discriminator of PcadvGAN server, and a client facial recognition model was ensembled simultaneously. The discriminator of the PcadvGAN server experienced mutation, crossover, and interaction with the ensemble model to reveal the optimal global weight of the PcadvGAN model. Finally, the global optimal aggregation parameter matrix of PcadvGAN was obtained by calculation. The server and the client shared the global optimal aggregation parameter matrix, enabling each client to generate private face images with high transferability and practicality. Targeted attack and non-targeted attack experiments demonstrated that the proposed method can generate high-quality, transferable, robust, private face images with only minor perturbations more effectively than other existing methods.