Exploring auxiliary context: discrete semantic transfer hashing for scalable image retrieval

Lei Zhu, Zi Huang, Zhihui Li, Liang Xie, Heng Tao Shen

Research output: Contribution to journalArticleResearchpeer-review

52 Citations (Scopus)

Abstract

Unsupervised hashing can desirably support scalable content-based image retrieval for its appealing advantages of semantic label independence, memory, and search efficiency. However, the learned hash codes are embedded with limited discriminative semantics due to the intrinsic limitation of image representation. To address the problem, in this paper, we propose a novel hashing approach, dubbed as discrete semantic transfer hashing (DSTH). The key idea is to directly augment the semantics of discrete image hash codes by exploring auxiliary contextual modalities. To this end, a unified hashing framework is formulated to simultaneously preserve visual similarities of images and perform semantic transfer from contextual modalities. Furthermore, to guarantee direct semantic transfer and avoid information loss, we explicitly impose the discrete constraint, bit-uncorrelation constraint, and bit-balance constraint on hash codes. A novel and effective discrete optimization method based on augmented Lagrangian multiplier is developed to iteratively solve the optimization problem. The whole learning process has linear computation complexity and desirable scalability. Experiments on three benchmark data sets demonstrate the superiority of DSTH compared with several state-of-the-art approaches.

Original languageEnglish
Article number8291840
Pages (from-to)5264-5276
Number of pages13
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume29
Issue number11
DOIs
Publication statusPublished - Nov 2018
Externally publishedYes

Keywords

  • Content-based image retrieval
  • discrete optimization
  • semantic transfer
  • unsupervised hashing
  • visual similarities

Cite this

@article{9aaf27806dae40d99c40d2c9801b99d0,
title = "Exploring auxiliary context: discrete semantic transfer hashing for scalable image retrieval",
abstract = "Unsupervised hashing can desirably support scalable content-based image retrieval for its appealing advantages of semantic label independence, memory, and search efficiency. However, the learned hash codes are embedded with limited discriminative semantics due to the intrinsic limitation of image representation. To address the problem, in this paper, we propose a novel hashing approach, dubbed as discrete semantic transfer hashing (DSTH). The key idea is to directly augment the semantics of discrete image hash codes by exploring auxiliary contextual modalities. To this end, a unified hashing framework is formulated to simultaneously preserve visual similarities of images and perform semantic transfer from contextual modalities. Furthermore, to guarantee direct semantic transfer and avoid information loss, we explicitly impose the discrete constraint, bit-uncorrelation constraint, and bit-balance constraint on hash codes. A novel and effective discrete optimization method based on augmented Lagrangian multiplier is developed to iteratively solve the optimization problem. The whole learning process has linear computation complexity and desirable scalability. Experiments on three benchmark data sets demonstrate the superiority of DSTH compared with several state-of-the-art approaches.",
keywords = "Content-based image retrieval, discrete optimization, semantic transfer, unsupervised hashing, visual similarities",
author = "Lei Zhu and Zi Huang and Zhihui Li and Liang Xie and Shen, {Heng Tao}",
year = "2018",
month = "11",
doi = "10.1109/TNNLS.2018.2797248",
language = "English",
volume = "29",
pages = "5264--5276",
journal = "IEEE Transactions on Neural Networks and Learning Systems",
issn = "2162-237X",
number = "11",

}

Exploring auxiliary context : discrete semantic transfer hashing for scalable image retrieval. / Zhu, Lei; Huang, Zi; Li, Zhihui; Xie, Liang; Shen, Heng Tao.

In: IEEE Transactions on Neural Networks and Learning Systems, Vol. 29, No. 11, 8291840, 11.2018, p. 5264-5276.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Exploring auxiliary context

T2 - discrete semantic transfer hashing for scalable image retrieval

AU - Zhu, Lei

AU - Huang, Zi

AU - Li, Zhihui

AU - Xie, Liang

AU - Shen, Heng Tao

PY - 2018/11

Y1 - 2018/11

N2 - Unsupervised hashing can desirably support scalable content-based image retrieval for its appealing advantages of semantic label independence, memory, and search efficiency. However, the learned hash codes are embedded with limited discriminative semantics due to the intrinsic limitation of image representation. To address the problem, in this paper, we propose a novel hashing approach, dubbed as discrete semantic transfer hashing (DSTH). The key idea is to directly augment the semantics of discrete image hash codes by exploring auxiliary contextual modalities. To this end, a unified hashing framework is formulated to simultaneously preserve visual similarities of images and perform semantic transfer from contextual modalities. Furthermore, to guarantee direct semantic transfer and avoid information loss, we explicitly impose the discrete constraint, bit-uncorrelation constraint, and bit-balance constraint on hash codes. A novel and effective discrete optimization method based on augmented Lagrangian multiplier is developed to iteratively solve the optimization problem. The whole learning process has linear computation complexity and desirable scalability. Experiments on three benchmark data sets demonstrate the superiority of DSTH compared with several state-of-the-art approaches.

AB - Unsupervised hashing can desirably support scalable content-based image retrieval for its appealing advantages of semantic label independence, memory, and search efficiency. However, the learned hash codes are embedded with limited discriminative semantics due to the intrinsic limitation of image representation. To address the problem, in this paper, we propose a novel hashing approach, dubbed as discrete semantic transfer hashing (DSTH). The key idea is to directly augment the semantics of discrete image hash codes by exploring auxiliary contextual modalities. To this end, a unified hashing framework is formulated to simultaneously preserve visual similarities of images and perform semantic transfer from contextual modalities. Furthermore, to guarantee direct semantic transfer and avoid information loss, we explicitly impose the discrete constraint, bit-uncorrelation constraint, and bit-balance constraint on hash codes. A novel and effective discrete optimization method based on augmented Lagrangian multiplier is developed to iteratively solve the optimization problem. The whole learning process has linear computation complexity and desirable scalability. Experiments on three benchmark data sets demonstrate the superiority of DSTH compared with several state-of-the-art approaches.

KW - Content-based image retrieval

KW - discrete optimization

KW - semantic transfer

KW - unsupervised hashing

KW - visual similarities

UR - http://www.scopus.com/inward/record.url?scp=85042118428&partnerID=8YFLogxK

U2 - 10.1109/TNNLS.2018.2797248

DO - 10.1109/TNNLS.2018.2797248

M3 - Article

C2 - 29994644

AN - SCOPUS:85042118428

VL - 29

SP - 5264

EP - 5276

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

SN - 2162-237X

IS - 11

M1 - 8291840

ER -