Differentially Private Deep Learning with Iterative Gradient Descent Optimization

Abstract

Deep learning has achieved great success in various areas and its success is closely linked to the availability of massive data. But in general, a large dataset could include sensitive data and therefore the model should have the capability to avoid privacy leakage. To achieve this aim, many works apply the famous privacy framework named differential privacy into deep learning to preserve privacy. In this article, we propose a novel perturbed iterative gradient descent optimization (PIGDO) algorithm and prove that this algorithm satisfies the differential privacy. Besides, we propose a modified moments accountant (MMA) method to conduct the privacy analysis and obtain a tighter bound of privacy loss compared with the original moments accountant method. A number of experiments demonstrate that our optimization algorithm can not only improve the model accuracy and training speed, but also achieve better privacy guarantees over the state-of-the-art algorithm while reaching the equivalent accuracy. We provide codes for all of our experiments in https://github.com/CGCL-codes/DPDLIGDO.git .