Schatten p-norm based Image-to-video adaptation for video action recognition

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


Human action recognition has been receiving extensive interest among researchers from the computer vision community. Numerous successful action recognition techniques have demonstrated the effectiveness of learning action knowledge from still images or motion videos. The relevant action information learned for the same action via various media types, such as images or videos, may be correlated. Nonetheless, less attention has been paid to adapting the action knowledge from images to videos to enhance action recognition performance in videos. Furthermore, most existing video action recognition methods suffer from insufficient labeled training videos. Overfitting could be an issue in these circumstances; hence, action recognition performance can be inhibited. This paper proposes an adaptation framework to transfer knowledge from images to videos for action recognition. A multi-task learning framework is designed to optimize the image and video domain classifiers jointly. The general Schatten p-norm is applied to the classifiers to mine the shared knowledge between these two domains. In this way, our framework can learn the correlated action semantics by leveraging the shared components of labeled images and videos. Our proposed approach can fully use the action knowledge from images and performs better in the case of poor and limited video data compared with the existing state-of-the-art action recognition techniques.

Original languageEnglish
Title of host publication2023 International Joint Conference on Neural Networks (IJCNN) Proceedings
EditorsLipo Wang, Teresa Ludermir, Tom Gedeon
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Electronic)9781665488679
ISBN (Print)9781665488686
Publication statusPublished - 2023
EventIEEE International Joint Conference on Neural Networks 2023 - Gold Coast, Australia
Duration: 18 Jun 202323 Jun 2023 (Proceedings) (Proceedings)


ConferenceIEEE International Joint Conference on Neural Networks 2023
Abbreviated titleIJCNN 2023
CityGold Coast
Internet address


  • Action recognition
  • adaptation
  • computer vision
  • general Schatten p-norm

Cite this