Incremental multimodal query construction for video search

Shicheng Xu; Huan Li; Xiaojun Chang; Shoou-I Yu; Xingzhong Du; Xuanchong Li; Lu Jiang; Zexi Mao; Zhenzhong Lan; Susanne Burger; Alexander Hauptmann

doi:10.1145/2671188.2749413

Incremental multimodal query construction for video search

Shicheng Xu, Huan Li, Xiaojun Chang, Shoou-I Yu, Xingzhong Du, Xuanchong Li, Lu Jiang, Zexi Mao, Zhenzhong Lan, Susanne Burger, Alexander Hauptmann

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other

13 Citations (Scopus)

Abstract

Recent improvements in content-based video search have led to systems with promising accuracy, thus opening up the possibility for interactive content-based video search to the general public. We present an interactive system based on a state-of-the-art content-based video search pipeline which enables users to do multimodal text-to-video and video-tovideo search in large video collections, and to incrementally refine queries through relevance feedback and model visualization. Also, the comprehensive functionalities enhance a flexible formulation of multimodal queries with different characteristics. Quantitative and qualitative analysis shows that our system is capable of assisting users to incrementally build effective queries over complex event topics.

Original language	English
Title of host publication	ICMR’15 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval
Subtitle of host publication	June 23–26, 2015 Shanghai, China
Editors	Yu-Gang Jiang, Cees Snoek, Nuno Vasconcelos
Place of Publication	New York NY USA
Publisher	Association for Computing Machinery (ACM)
Pages	675-678
Number of pages	4
ISBN (Electronic)	9781450332743
DOIs	https://doi.org/10.1145/2671188.2749413
Publication status	Published - 2015
Externally published	Yes
Event	ACM International Conference on Multimedia Retrieval 2015 - Shanghai, China Duration: 23 Jun 2015 → 26 Jun 2015 Conference number: 5th https://dl.acm.org/doi/proceedings/10.1145/2671188 (Proceedings)

Conference

Conference	ACM International Conference on Multimedia Retrieval 2015
Abbreviated title	ICMR 2015
Country	China
City	Shanghai
Period	23/06/15 → 26/06/15
Internet address	https://dl.acm.org/doi/proceedings/10.1145/2671188 (Proceedings)

Keywords

Content-based video search
Multimedia event detection
Multimodal query generation

Access to Document

10.1145/2671188.2749413

Link to publication in Scopus

Cite this

Xu, S., Li, H., Chang, X., Yu, S-I., Du, X., Li, X., Jiang, L., Mao, Z., Lan, Z., Burger, S., & Hauptmann, A. (2015). Incremental multimodal query construction for video search. In Y-G. Jiang, C. Snoek, & N. Vasconcelos (Eds.), ICMR’15 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval: June 23–26, 2015 Shanghai, China (pp. 675-678). Association for Computing Machinery (ACM). https://doi.org/10.1145/2671188.2749413

Xu, Shicheng ; Li, Huan ; Chang, Xiaojun ; Yu, Shoou-I ; Du, Xingzhong ; Li, Xuanchong ; Jiang, Lu ; Mao, Zexi ; Lan, Zhenzhong ; Burger, Susanne ; Hauptmann, Alexander. / Incremental multimodal query construction for video search. ICMR’15 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval: June 23–26, 2015 Shanghai, China. editor / Yu-Gang Jiang ; Cees Snoek ; Nuno Vasconcelos. New York NY USA : Association for Computing Machinery (ACM), 2015. pp. 675-678

@inproceedings{4873d92cec4241999e3970c805c624f7,

title = "Incremental multimodal query construction for video search",

abstract = "Recent improvements in content-based video search have led to systems with promising accuracy, thus opening up the possibility for interactive content-based video search to the general public. We present an interactive system based on a state-of-the-art content-based video search pipeline which enables users to do multimodal text-to-video and video-tovideo search in large video collections, and to incrementally refine queries through relevance feedback and model visualization. Also, the comprehensive functionalities enhance a flexible formulation of multimodal queries with different characteristics. Quantitative and qualitative analysis shows that our system is capable of assisting users to incrementally build effective queries over complex event topics.",

keywords = "Content-based video search, Multimedia event detection, Multimodal query generation",

author = "Shicheng Xu and Huan Li and Xiaojun Chang and Shoou-I Yu and Xingzhong Du and Xuanchong Li and Lu Jiang and Zexi Mao and Zhenzhong Lan and Susanne Burger and Alexander Hauptmann",

year = "2015",

doi = "10.1145/2671188.2749413",

language = "English",

pages = "675--678",

editor = "Jiang, {Yu-Gang } and Snoek, {Cees } and Vasconcelos, {Nuno }",

booktitle = "ICMR{\textquoteright}15 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval",

publisher = "Association for Computing Machinery (ACM)",

address = "United States of America",

note = "ACM International Conference on Multimedia Retrieval 2015, ICMR 2015 ; Conference date: 23-06-2015 Through 26-06-2015",

url = "https://dl.acm.org/doi/proceedings/10.1145/2671188",

}

Xu, S, Li, H, Chang, X, Yu, S-I, Du, X, Li, X, Jiang, L, Mao, Z, Lan, Z, Burger, S & Hauptmann, A 2015, Incremental multimodal query construction for video search. in Y-G Jiang, C Snoek & N Vasconcelos (eds), ICMR’15 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval: June 23–26, 2015 Shanghai, China. Association for Computing Machinery (ACM), New York NY USA, pp. 675-678, ACM International Conference on Multimedia Retrieval 2015, Shanghai, China, 23/06/15. https://doi.org/10.1145/2671188.2749413

Incremental multimodal query construction for video search. / Xu, Shicheng; Li, Huan; Chang, Xiaojun; Yu, Shoou-I; Du, Xingzhong; Li, Xuanchong; Jiang, Lu; Mao, Zexi; Lan, Zhenzhong; Burger, Susanne; Hauptmann, Alexander.

ICMR’15 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval: June 23–26, 2015 Shanghai, China. ed. / Yu-Gang Jiang; Cees Snoek; Nuno Vasconcelos. New York NY USA : Association for Computing Machinery (ACM), 2015. p. 675-678.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other

TY - GEN

T1 - Incremental multimodal query construction for video search

AU - Xu, Shicheng

AU - Li, Huan

AU - Chang, Xiaojun

AU - Yu, Shoou-I

AU - Du, Xingzhong

AU - Li, Xuanchong

AU - Jiang, Lu

AU - Mao, Zexi

AU - Lan, Zhenzhong

AU - Burger, Susanne

AU - Hauptmann, Alexander

N1 - Conference code: 5th

PY - 2015

Y1 - 2015

N2 - Recent improvements in content-based video search have led to systems with promising accuracy, thus opening up the possibility for interactive content-based video search to the general public. We present an interactive system based on a state-of-the-art content-based video search pipeline which enables users to do multimodal text-to-video and video-tovideo search in large video collections, and to incrementally refine queries through relevance feedback and model visualization. Also, the comprehensive functionalities enhance a flexible formulation of multimodal queries with different characteristics. Quantitative and qualitative analysis shows that our system is capable of assisting users to incrementally build effective queries over complex event topics.

AB - Recent improvements in content-based video search have led to systems with promising accuracy, thus opening up the possibility for interactive content-based video search to the general public. We present an interactive system based on a state-of-the-art content-based video search pipeline which enables users to do multimodal text-to-video and video-tovideo search in large video collections, and to incrementally refine queries through relevance feedback and model visualization. Also, the comprehensive functionalities enhance a flexible formulation of multimodal queries with different characteristics. Quantitative and qualitative analysis shows that our system is capable of assisting users to incrementally build effective queries over complex event topics.

KW - Content-based video search

KW - Multimedia event detection

KW - Multimodal query generation

UR - http://www.scopus.com/inward/record.url?scp=84962439669&partnerID=8YFLogxK

U2 - 10.1145/2671188.2749413

DO - 10.1145/2671188.2749413

M3 - Conference Paper

AN - SCOPUS:84962439669

SP - 675

EP - 678

BT - ICMR’15 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval

A2 - Jiang, Yu-Gang

A2 - Snoek, Cees

A2 - Vasconcelos, Nuno

PB - Association for Computing Machinery (ACM)

CY - New York NY USA

T2 - ACM International Conference on Multimedia Retrieval 2015

Y2 - 23 June 2015 through 26 June 2015

ER -

Xu S, Li H, Chang X, Yu S-I, Du X, Li X et al. Incremental multimodal query construction for video search. In Jiang Y-G, Snoek C, Vasconcelos N, editors, ICMR’15 - Proceedings of the 2015 ACM International Conference on Multimedia Retrieval: June 23–26, 2015 Shanghai, China. New York NY USA: Association for Computing Machinery (ACM). 2015. p. 675-678 https://doi.org/10.1145/2671188.2749413