Weakly supervised fine-grained categorization with part-based image representation

Yu Zhang, Xiu Shen Wei, Jianxin Wu, Jianfei Cai, Jiangbo Lu, Viet Anh Nguyen, Minh N. Do

Research output: Contribution to journalArticleResearchpeer-review

64 Citations (Scopus)

Abstract

In this paper, we propose a fine-grained image categorization system with easy deployment. We do not use any object/part annotation (weakly supervised) in the training or in the testing stage, but only class labels for training images. Fine-grained image categorization aims to classify objects with only subtle distinctions (e.g., two breeds of dogs that look alike). Most existing works heavily rely on object/part detectors to build the correspondence between object parts, which require accurate object or object part annotations at least for training images. The need for expensive object annotations prevents the wide usage of these methods. Instead, we propose to generate multi-scale part proposals from object proposals, select useful part proposals, and use them to compute a global image representation for categorization. This is specially designed for the weakly supervised fine-grained categorization task, because useful parts have been shown to play a critical role in existing annotation-dependent works, but accurate part detectors are hard to acquire. With the proposed image representation, we can further detect and visualize the key (most discriminative) parts in objects of different classes. In the experiments, the proposed weakly supervised method achieves comparable or better accuracy than the state-of-the-art weakly supervised methods and most existing annotation-dependent methods on three challenging datasets. Its success suggests that it is not always necessary to learn expensive object/part detectors in fine-grained image categorization.

Original languageEnglish
Pages (from-to)1713-1725
Number of pages13
JournalIEEE Transactions on Image Processing
Volume25
Issue number4
DOIs
Publication statusPublished - Apr 2016
Externally publishedYes

Keywords

  • Fine grained categorization
  • part selection
  • weakly supervised

Cite this

Zhang, Yu ; Wei, Xiu Shen ; Wu, Jianxin ; Cai, Jianfei ; Lu, Jiangbo ; Nguyen, Viet Anh ; Do, Minh N. / Weakly supervised fine-grained categorization with part-based image representation. In: IEEE Transactions on Image Processing. 2016 ; Vol. 25, No. 4. pp. 1713-1725.
@article{d45a17bdcf724f688604c27eb71cc2fa,
title = "Weakly supervised fine-grained categorization with part-based image representation",
abstract = "In this paper, we propose a fine-grained image categorization system with easy deployment. We do not use any object/part annotation (weakly supervised) in the training or in the testing stage, but only class labels for training images. Fine-grained image categorization aims to classify objects with only subtle distinctions (e.g., two breeds of dogs that look alike). Most existing works heavily rely on object/part detectors to build the correspondence between object parts, which require accurate object or object part annotations at least for training images. The need for expensive object annotations prevents the wide usage of these methods. Instead, we propose to generate multi-scale part proposals from object proposals, select useful part proposals, and use them to compute a global image representation for categorization. This is specially designed for the weakly supervised fine-grained categorization task, because useful parts have been shown to play a critical role in existing annotation-dependent works, but accurate part detectors are hard to acquire. With the proposed image representation, we can further detect and visualize the key (most discriminative) parts in objects of different classes. In the experiments, the proposed weakly supervised method achieves comparable or better accuracy than the state-of-the-art weakly supervised methods and most existing annotation-dependent methods on three challenging datasets. Its success suggests that it is not always necessary to learn expensive object/part detectors in fine-grained image categorization.",
keywords = "Fine grained categorization, part selection, weakly supervised",
author = "Yu Zhang and Wei, {Xiu Shen} and Jianxin Wu and Jianfei Cai and Jiangbo Lu and Nguyen, {Viet Anh} and Do, {Minh N.}",
year = "2016",
month = "4",
doi = "10.1109/TIP.2016.2531289",
language = "English",
volume = "25",
pages = "1713--1725",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
number = "4",

}

Weakly supervised fine-grained categorization with part-based image representation. / Zhang, Yu; Wei, Xiu Shen; Wu, Jianxin; Cai, Jianfei; Lu, Jiangbo; Nguyen, Viet Anh; Do, Minh N.

In: IEEE Transactions on Image Processing, Vol. 25, No. 4, 04.2016, p. 1713-1725.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Weakly supervised fine-grained categorization with part-based image representation

AU - Zhang, Yu

AU - Wei, Xiu Shen

AU - Wu, Jianxin

AU - Cai, Jianfei

AU - Lu, Jiangbo

AU - Nguyen, Viet Anh

AU - Do, Minh N.

PY - 2016/4

Y1 - 2016/4

N2 - In this paper, we propose a fine-grained image categorization system with easy deployment. We do not use any object/part annotation (weakly supervised) in the training or in the testing stage, but only class labels for training images. Fine-grained image categorization aims to classify objects with only subtle distinctions (e.g., two breeds of dogs that look alike). Most existing works heavily rely on object/part detectors to build the correspondence between object parts, which require accurate object or object part annotations at least for training images. The need for expensive object annotations prevents the wide usage of these methods. Instead, we propose to generate multi-scale part proposals from object proposals, select useful part proposals, and use them to compute a global image representation for categorization. This is specially designed for the weakly supervised fine-grained categorization task, because useful parts have been shown to play a critical role in existing annotation-dependent works, but accurate part detectors are hard to acquire. With the proposed image representation, we can further detect and visualize the key (most discriminative) parts in objects of different classes. In the experiments, the proposed weakly supervised method achieves comparable or better accuracy than the state-of-the-art weakly supervised methods and most existing annotation-dependent methods on three challenging datasets. Its success suggests that it is not always necessary to learn expensive object/part detectors in fine-grained image categorization.

AB - In this paper, we propose a fine-grained image categorization system with easy deployment. We do not use any object/part annotation (weakly supervised) in the training or in the testing stage, but only class labels for training images. Fine-grained image categorization aims to classify objects with only subtle distinctions (e.g., two breeds of dogs that look alike). Most existing works heavily rely on object/part detectors to build the correspondence between object parts, which require accurate object or object part annotations at least for training images. The need for expensive object annotations prevents the wide usage of these methods. Instead, we propose to generate multi-scale part proposals from object proposals, select useful part proposals, and use them to compute a global image representation for categorization. This is specially designed for the weakly supervised fine-grained categorization task, because useful parts have been shown to play a critical role in existing annotation-dependent works, but accurate part detectors are hard to acquire. With the proposed image representation, we can further detect and visualize the key (most discriminative) parts in objects of different classes. In the experiments, the proposed weakly supervised method achieves comparable or better accuracy than the state-of-the-art weakly supervised methods and most existing annotation-dependent methods on three challenging datasets. Its success suggests that it is not always necessary to learn expensive object/part detectors in fine-grained image categorization.

KW - Fine grained categorization

KW - part selection

KW - weakly supervised

UR - http://www.scopus.com/inward/record.url?scp=84964246694&partnerID=8YFLogxK

U2 - 10.1109/TIP.2016.2531289

DO - 10.1109/TIP.2016.2531289

M3 - Article

C2 - 26890872

AN - SCOPUS:84964246694

VL - 25

SP - 1713

EP - 1725

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 4

ER -