Accelerating TEE-based DNN inference using mean shift network pruning

Chengyao Xu, Shangqi Lai

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


In recent years, deep neural networks (DNNs) have achieved great success in many areas and have been deployed as cloud services to bring convenience to people’s daily lives. However, the widespread use of DNNs in the cloud brings critical privacy concerns. Researchers have proposed many solutions to address the privacy concerns of deploying DNN in the cloud, and one major category of solutions rely on a trusted execution environment (TEE). Nonetheless, the DNN inference requires extensive memory and computing resources to achieve accurate decision-making, which does not operate well in TEE with restricted memory space. This paper proposes a network pruning algorithm based on mean shift clustering to reduce the model size and improve the inference performance in TEE. The core idea of our design is to use a mean shift algorithm to aggregate the weight values automatically and prune the network based on the distance between the weight and center. Our experiments prune three popular networks on the CIFAR-10 dataset. The experimental results show that our algorithm successfully reduces the network size without affecting its accuracy. The inference in TEE is accelerated by 20%.

Original languageEnglish
Title of host publication17th EAI International Conference, QShine 2021 Virtual Event, November 29–30, 2021 Proceedings
EditorsXingliang Yuan, Wei Bao, Xun Yi, Nguyen Hoang Tran
Place of PublicationCham Switzerland
Number of pages17
ISBN (Electronic)9783030914240
ISBN (Print)9783030914233
Publication statusPublished - 2021
EventEAI International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness 2021 - Online, Melbourne, Australia
Duration: 29 Nov 202130 Nov 2021
Conference number: 17th (Website) (Proceedings)

Publication series

NameLecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST
Volume402 LNICST
ISSN (Print)1867-8211
ISSN (Electronic)1867-822X


ConferenceEAI International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness 2021
Abbreviated titleQShine 2021
Internet address

Cite this