Efficient video object co-localization with co-saliency activated tracklets

Koteswar Rao Jerripothula, Jianfei Cai, Junsong Yuan

Research output: Contribution to journalArticleResearchpeer-review

25 Citations (Scopus)


Video object co-localization is the task of jointly localizing common visual objects across videos. Due to the large variations both across the videos and within each video, it is quite challenging to identify and track the common objects jointly. Unlike the previous joint frameworks that use a large number of bounding box proposals to attack the problem, we propose to leverage co-saliency activated tracklets to efficiently address the problem. To highlight the common object regions, we first explore inter-video commonness, intra-video commonness, and motion saliency to generate the co-saliency maps for a small number of selected key frames at regular intervals. Object proposals of high objectness and co-saliency scores in those frames are tracked across each interval to build tracklets. Finally, the best tube for a video is obtained through selecting the optimal tracklet from each interval with the help of confidence and smoothness constraints. Experimental results on the benchmark YouTube-objects dataset show that the proposed method outperforms the state-of-the-art methods in terms of accuracy and speed under both weakly supervised and unsupervised settings. Moreover, by noticing the existing benchmark dataset lacks of sufficient annotations for object localization (only one annotated frame per video), we further annotate more than 15k frames of the YouTube videos and develop a new benchmark dataset for video co-localization.

Original languageEnglish
Article number8290832
Pages (from-to)744-755
Number of pages12
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number3
Publication statusPublished - Mar 2019
Externally publishedYes


  • co-localization
  • co-saliency
  • Tracklets
  • video

Cite this