Action proposals using hierarchical clustering of super-trajectories

Tianyi Zhang, Li Niu, Jianfei Cai, Alex C. Kot

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


Action localization aims to determine the spatial and temporal location of certain action which appears in a video. To facilitate action localization, spatio-temporal proposals which are likely to contain the action of interest are extracted to reduce the search space of candidate locations in a video, inspired by the object proposals in images. In this paper, considering the effectiveness of spatio-temporal trajectories for video action recognition and action proposal generation, we build our unsupervised action proposal generation pipeline upon super-trajectories. Specifically, we first group trajectories into super-trajectories inspired by super-voxels, and then employ hierarchical clustering on super-trajectories by taking different aspect and temporal ratios into consideration. Comprehensive experiments on two benchmark datasets (i.e., UCF-sports and MSR-II) demonstrate that our action proposal generation pipeline not only achieves the state-of-the-art recall, but also achieves competitive results for the action localization task.

Original languageEnglish
Title of host publication2017 IEEE Visual Communications and Image Processing (VCIP 2017)
EditorsEckehard Steinbach, Weisi Lin, Carlos Busso, Siwei Ma
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages4
ISBN (Electronic)9781538604625
ISBN (Print)9781538604632
Publication statusPublished - 2017
Externally publishedYes
EventIEEE Visual Communications and Image Processing 2017 - St Petersburg, United States of America
Duration: 10 Dec 201713 Dec 2017 (Proceedings)


ConferenceIEEE Visual Communications and Image Processing 2017
Abbreviated titleVCIP 2017
Country/TerritoryUnited States of America
CitySt Petersburg
OtherIEEE Visual Communications and Image Processing
Internet address


  • Action Localization
  • Action Proposal
  • Fisher Vector
  • Trajectory Grouping

Cite this