TY - JOUR
T1 - Semantics-preserving graph propagation for Zero-Shot Object Detection
AU - Yan, Caixia
AU - Zheng, Qinghua
AU - Chang, Xiaojun
AU - Luo, Minnan
AU - Yeh, Chung Hsing
AU - Hauptman, Alexander G.
PY - 2020/7/30
Y1 - 2020/7/30
N2 - Most existing object detection models are restricted to detecting objects from previously seen categories, an approach that tends to become infeasible for rare or novel concepts. Accordingly, in this paper, we explore object detection in the context of zero-shot learning, i.e., Zero-Shot Object Detection (ZSD), to concurrently recognize and localize objects from novel concepts. Existing ZSD algorithms are typically based on a strict mapping-transfer strategy that suffers from a significant visual-semantic gap. To bridge the gap, we propose a novel Semantics-Preserving Graph Propagation model for ZSD based on Graph Convolutional Networks (GCN). More specifically, we develop a graph construction module to flexibly build category graphs by leveraging diverse correlations between category nodes; this is followed by two semantics-preserving graph propagation modules that enhance both category and region representations. Benefiting from the multi-step graph propagation process, both the semantic description and structural knowledge exhibited in prior category graphs can be effectively leveraged to boost the generalization capability of the learned projection function. Experiments on existing seen/unseen splits of three popular object detection datasets demonstrate that the proposed approach performs favorably against state-of-the-art ZSD methods.
AB - Most existing object detection models are restricted to detecting objects from previously seen categories, an approach that tends to become infeasible for rare or novel concepts. Accordingly, in this paper, we explore object detection in the context of zero-shot learning, i.e., Zero-Shot Object Detection (ZSD), to concurrently recognize and localize objects from novel concepts. Existing ZSD algorithms are typically based on a strict mapping-transfer strategy that suffers from a significant visual-semantic gap. To bridge the gap, we propose a novel Semantics-Preserving Graph Propagation model for ZSD based on Graph Convolutional Networks (GCN). More specifically, we develop a graph construction module to flexibly build category graphs by leveraging diverse correlations between category nodes; this is followed by two semantics-preserving graph propagation modules that enhance both category and region representations. Benefiting from the multi-step graph propagation process, both the semantic description and structural knowledge exhibited in prior category graphs can be effectively leveraged to boost the generalization capability of the learned projection function. Experiments on existing seen/unseen splits of three popular object detection datasets demonstrate that the proposed approach performs favorably against state-of-the-art ZSD methods.
KW - graph propagation
KW - semantic embedding
KW - semantic relation
KW - Zero-shot object detection
UR - http://www.scopus.com/inward/record.url?scp=85090340166&partnerID=8YFLogxK
U2 - 10.1109/TIP.2020.3011807
DO - 10.1109/TIP.2020.3011807
M3 - Article
AN - SCOPUS:85090340166
VL - 29
SP - 8163
EP - 8176
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
SN - 1057-7149
ER -