PrivateDL: privacy-preserving collaborative deep learning against leakage from gradient sharing

Qi Zhao, Chuan Zhao, Shujie Cui, Shan Jing, Zhenxiang Chen

Research output: Contribution to journalArticleResearchpeer-review

5 Citations (Scopus)

Abstract

Large-scale data training is vital to the generalization performance of deep learning (DL) models. However, collecting data directly is associated with increased risk of privacy disclosure, particularly in special fields such as healthcare, finance, and genomics. To protect training data privacy, collaborative deep learning (CDL) has been proposed to enable joint training from multiple data owners while providing reliable privacy guarantee. However, recent studies have shown that CDL is vulnerable to several attacks that could reveal sensitive information about the original training data. One of the most powerful attacks benefits from the leakage from gradient sharing during collaborative training process. In this study, we present a new CDL framework, (Formula presented.), to effectively protect private training data against leakage from gradient sharing. Unlike conventional training process that trains on private data directly, (Formula presented.) allows effective transfer of relational knowledge from sensitive data to public data in a privacy-preserving way, and enables participants to jointly learn local models based on the public data with noise-preserving labels. This way, (Formula presented.) establishes a privacy gap between the local models and the private datasets, thereby ensuring privacy against the attacks launched to the local models through gradient sharing. Moreover, we propose a new algorithm called Distributed Aggregation Stochastic Gradient Descent, which is designed to improve the efficiency and accuracy of CDL, especially in the asynchronous training mode. Experimental results demonstrate that (Formula presented.) preserves data privacy with reasonable performance overhead.

Original languageEnglish
Pages (from-to)1262-1279
Number of pages18
JournalInternational Journal of Intelligent Systems
Volume35
Issue number8
DOIs
Publication statusPublished - Aug 2020
Externally publishedYes

Keywords

  • collaborative deep learning
  • gradient sharing
  • machine learning
  • privacy-preserving technique

Cite this