Privacy-Preserving Deep Learning Framework Based on Restricted Boltzmann Machines and Instance Reduction Algorithms
-
Published:2024-02-01
Issue:3
Volume:14
Page:1224
-
ISSN:2076-3417
-
Container-title:Applied Sciences
-
language:en
-
Short-container-title:Applied Sciences
Author:
Alshammari Alya1, El Hindi Khalil1ORCID
Affiliation:
1. Department of Computer Science, College of Computer & Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
Abstract
The combination of collaborative deep learning and Cyber-Physical Systems (CPSs) has the potential to improve decision-making, adaptability, and efficiency in dynamic and distributed environments. However, it brings privacy, communication, and resource restrictions concerns that must be properly addressed for successful implementation in real-world CPS systems. Various privacy-preserving techniques have been proposed, but they often add complexity and decrease accuracy and utility. In this paper, we propose a privacy-preserving deep learning framework that combines Instance Reduction Techniques (IR) and the Restricted Boltzmann Machine (RBM) to preserve privacy while overcoming the limitations of other frameworks. The RBM encodes training data to retain relevant features, and IR selects the relevant encoded instances to send to the server for training. Privacy is preserved because only a small subset of the training data is sent to the server. Moreover, it is sent after encoding it using RBM. Experiments show that our framework preserves privacy with little loss of accuracy and a substantial reduction in training time. For example, using our framework, a CNN model for the MNIST dataset achieves 96% accuracy compared to 99% in a standard collaborative framework (with no privacy measures taken), with training time reduced from 133.259 s to 99.391 s. Our MLP model for MNIST achieves 97% accuracy compared to 98% in the standard collaborative framework, with training time reduced from 118.146 s to 87.873 s. Compared to other studies, our method is a simple approach that protects privacy, maintains the utility of deep learning models, and reduces training time and communication costs.
Funder
King Saud University, Riyadh, Saudi Arabia
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference76 articles.
1. Shokri, R., and Shmatikov, V. (October, January 29). Privacy-Preserving Deep Learning. Proceedings of the 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA. 2. Fredrikson, M., Jha, S., and Ristenpart, T. (2015, January 12–16). Model Inversion Attacks That Exploit Confidence Information and Basic Countermeasures. Proceedings of the ACM Conference on Computer and Communications Security, Denver, CO, USA. 3. Carlini, N., Tramèr, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., and Erlingsson, Ú. (2021, January 11–13). Extracting Training Data from Large Language Models. Proceedings of the 30th USENIX Security Symposium, Vancouver, BC, Canada. 4. Wang, B., and Gong, N.Z. (2018, January 21–23). Stealing Hyperparameters in Machine Learning. Proceedings of the IEEE Symposium on Security and Privacy, San Francisco, CA, USA. 5. Tramèr, F., Zhang, F., Juels, A., Reiter, M.K., and Ristenpart, T. (2016, January 10–12). Stealing Machine Learning Models via Prediction APIs. Proceedings of the 25th USENIX Security Symposium, Austin, TX, USA.
|
|