ZSTAD: zero-shot temporal activity detection

Lingling Zhang, Xiaojun Chang, Jun Liu, Minnan Luo, Sen Wang, Zongyuan Ge, Alexander Hauptmann

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

4 Citations (Scopus)

Abstract

An integral part of video analysis and surveillance is temporal activity detection, which means to simultaneously recognize and localize activities in long untrimmed videos. Currently, the most effective methods of temporal activity detection are based on deep learning, and they typically perform very well with large scale annotated videos for training. However, these methods are limited in real applications due to the unavailable videos about certain activity classes and the time-consuming data annotation. To solve this challenging problem, we propose a novel task setting called zero-shot temporal activity detection (ZSTAD), where activities that have never been seen in training can still be detected. We design an end-To-end deep network based on R-C3D as the architecture for this solution. The proposed network is optimized with an innovative loss function that considers the embeddings of activity labels and their super-classes while learning the common semantics of seen and unseen activities. Experiments on both the THUMOS'14 and the Charades datasets show promising performance in terms of detecting unseen activities.

Original languageEnglish
Title of host publicationProceedings - 33th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2020
EditorsCe Liu, Greg Mori, Kate Saenko, Silvio Savarese
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages876-885
Number of pages10
ISBN (Electronic)9781728171685
ISBN (Print)9781728171692
DOIs
Publication statusPublished - 2020
EventIEEE Conference on Computer Vision and Pattern Recognition 2020 - Virtual, China
Duration: 14 Jun 202019 Jun 2020
http://cvpr2020.thecvf.com (Website )
https://openaccess.thecvf.com/CVPR2020 (Proceedings)
https://ieeexplore.ieee.org/xpl/conhome/9142308/proceeding (Proceedings)

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075

Conference

ConferenceIEEE Conference on Computer Vision and Pattern Recognition 2020
Abbreviated titleCVPR 2020
CountryChina
CityVirtual
Period14/06/2019/06/20
Internet address

Cite this