Enabling secure and effective near-duplicate detection over encrypted in-network storage

Helei Cui, Xingliang Yuan, Yifeng Zheng, Cong Wang

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

27 Citations (Scopus)


Near-duplicate detection (NDD) plays an essential role for effective resource utilization and possible traffic alleviation in many emerging network architectures, leveraging in-network storage for various content-centric services. As innetwork storage grows, data security has become one major concern. Though encryption is viable for in-network data protection, current techniques are still lacking for effectively locating encrypted near-duplicate data, making the benefits of NDD practically invalidated. Besides, adopting encrypted innetwork storage further complicates the user authorization when locating near-duplicate data from multiple content providers under different keys. In this paper, we propose a secure and effective NDD system over encrypted in-network storage supporting multiple content providers. Our design bridges locality-sensitive hashing (LSH) with a newly developed cryptographic primitive, multi-key searchable encryption, which allows the user to send only one encrypted query to access near-duplicate data encrypted under different keys. It relieves the users from multiple rounds of interactions or sending multiple different queries respectively. As simply applying LSH does not ensure the detection quality, we then leverage Yao's garbled circuits to build a secure protocol to obtain highly accurate results, without user-side post-processing. We formally analyze the security strength. Experiments demonstrate our system achieves practical performance with comparable accuracy to plaintext.

Original languageEnglish
Title of host publicationIEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications
Subtitle of host publicationSan Francisco, California, USA 10-14 April 2016
EditorsSung-Ju Lee, Chiara Petrioli, Peng-Jun Wan
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages9
ISBN (Electronic)9781467399531
ISBN (Print)9781467399548
Publication statusPublished - 2016
Externally publishedYes
EventIEEE Conference on Computer Communications 2016 - San Francisco, United States of America
Duration: 10 Apr 201615 Apr 2016
Conference number: 35th


ConferenceIEEE Conference on Computer Communications 2016
Abbreviated titleIEEE INFOCOM 2016
Country/TerritoryUnited States of America
CitySan Francisco
Internet address

Cite this