Strategies for searching video content with text queries or video examples

Shoou I. Yu, Shicheng Xu, Zhigang Ma, Huan Li, Alexander G. Hauptmann, Xiaojun Chang, Yi Yang, Deyu Meng, Ming Lin, Zhenzhong Lan, Chuang Gan, Zhongwen Xu, Zexi Mao, Xuanchong Li, Lu Jiang, Xingzhong Du

Research output: Contribution to journalArticleResearchpeer-review

2 Citations (Scopus)

Abstract

The large number of user-generated videos uploaded on to the Internet everyday has led to many commercial video search engines, which mainly rely on text metadata for search. However, metadata is often lacking for user-generated videos, thus these videos are unsearchable by current search engines. Therefore, content-based video retrieval (CBVR) tackles this metadata-scarcity problem by directly analyzing the visual and audio streams of each video. CBVR encompasses multiple research topics, including low-level feature design, feature fusion, semantic detector training and video search/reranking. We present novel strategies in these topics to enhance CBVR in both accuracy and speed under different query inputs, including pure textual queries and query by video examples. Our proposed strategies have been incorporated into our submission for the TRECVID 2014 Multimedia Event Detection evaluation, where our system outperformed other submissions in both text queries and video example queries, thus demonstrating the effectiveness of our proposed approaches.

Original languageEnglish
Pages (from-to)227-238
Number of pages12
JournalITE Transactions on Media Technology and Applications
Volume4
Issue number3
DOIs
Publication statusPublished - 2016
Externally publishedYes

Keywords

  • Content-based Video Retrieval
  • Motion & Image Features
  • Multimedia Event Detection
  • Multimodal Fusion
  • Reranking
  • Semantic Concept Detectors

Cite this