Schatten p-norm based Image-to-video adaptation for video action recognition

Human action recognition has been receiving extensive interest among researchers from the computer vision community. Numerous successful action recognition techniques have demonstrated the effectiveness of learning action knowledge from still images or motion videos. The relevant action information learned for the same action via various media types, such as images or videos, may be correlated. Nonetheless, less attention has been paid to adapting the action knowledge from images to videos to enhance action recognition performance in videos. Furthermore, most existing video action recognition methods suffer from insufficient labeled training videos. Overfitting could be an issue in these circumstances; hence, action recognition performance can be inhibited. This paper proposes an adaptation framework to transfer knowledge from images to videos for action recognition. A multi-task learning framework is designed to optimize the image and video domain classifiers jointly. The general Schatten p-norm is applied to the classifiers to mine the shared knowledge between these two domains. In this way, our framework can learn the correlated action semantics by leveraging the shared components of labeled images and videos. Our proposed approach can fully use the action knowledge from images and performs better in the case of poor and limited video data compared with the existing state-of-the-art action recognition techniques.