Micro-video tagging via jointly modeling social influence and tag relation

Xiao Wang, Tian Gan, Yinwei Wei, Jianlong Wu, Dai Meng, Liqiang Nie

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

3 Citations (Scopus)

Abstract

The last decade has witnessed the proliferation of micro-videos on various user-generated content platforms. According to our statistics, around 85.7% of micro-videos lack annotation. In this paper, we focus on annotating micro-videos with tags. Existing methods mostly focus on analyzing video content, neglecting users' social influence and tag relation. Meanwhile, existing tag relation construction methods suffer from either deficient performance or low tag coverage. To jointly model social influence and tag relation, we formulate micro-video tagging as a link prediction problem in a constructed heterogeneous network. Specifically, the tag relation (represented by tag ontology) is constructed in a semi-supervised manner. Then, we combine tag relation, video-tag annotation, and user follow relation to build the network. Afterward, a better video and tag representation are derived through Behavior Spread modeling and visual and linguistic knowledge aggregation. Finally, the semantic similarity between each micro-video and all candidate tags is calculated in this video-tag network. Extensive experiments on industrial datasets of three verticals verify the superiority of our model compared with several state-of-the-art baselines.

Original languageEnglish
Title of host publicationProceedings of the 30th ACM International Conference on Multimedia
EditorsMarco Bertini, Klaus Schoeffmann
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages4478-4486
Number of pages9
ISBN (Electronic)9781450392037
DOIs
Publication statusPublished - 2022
Externally publishedYes
EventACM International Conference on Multimedia 2022 - Lisbon, Portugal
Duration: 10 Oct 202214 Oct 2022
Conference number: 30th
https://dl.acm.org/doi/proceedings/10.1145/3503161 (Proceedings)
https://2022.acmmm.org/ (Website)

Conference

ConferenceACM International Conference on Multimedia 2022
Abbreviated titleMM 2022
Country/TerritoryPortugal
CityLisbon
Period10/10/2214/10/22
Internet address

Keywords

  • behavior spread
  • micro-video tagging
  • ontology construction

Cite this