Adversarial action data augmentation for similar gesture action recognition

DI Wu, Junjun Chen, Nabin Sharma, Shirui Pan, Guodong Long, Michael Blumenstein

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Human gestures are unique for recognizing and describing human actions, and video-based human action recognition techniques are effective solutions to varies real-world applications, such as surveillance, video indexing, and human-computer interaction. Most existing video human action recognition approaches either using handcraft features from the frames or deep learning models such as convolutional neural networks (CNN) and recurrent neural networks (RNN); however, they have mostly overlooked the similar gestures between different actions when processing the frames into the models. The classifiers suffer from similar features extracted from similar gestures, which are unable to classify the actions in the video streams. In this paper, we propose a novel framework with generative adversarial networks (GAN) to generate the data augmentation for similar gesture action recognition. The contribution of our work is tri-fold: 1) we proposed a novel action data augmentation framework (ADAF) to enlarge the differences between the actions with very similar gestures; 2) the framework can boost the classification performance either on similar gesture action pairs or the whole dataset; 3) experiments conducted on both KTH and UCF101 datasets show that our data augmentation framework boost the performance on both similar gestures actions as well as the whole dataset compared with baseline methods such as 2DCNN and 3DCNN.

Original languageEnglish
Title of host publicationInternational Joint Conference on Neural Networks (IJCNN) 2019
EditorsPlamen Angelov, Manuel Roveri
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Electronic)9781728119854
ISBN (Print)9781728119861
DOIs
Publication statusPublished - 2019
EventIEEE International Joint Conference on Neural Networks 2019 - Budapest, Hungary
Duration: 14 Jul 201919 Jul 2019
https://www.ijcnn.org/

Conference

ConferenceIEEE International Joint Conference on Neural Networks 2019
Abbreviated titleIJCNN 2019
CountryHungary
CityBudapest
Period14/07/1919/07/19
Internet address

Keywords

  • Action recognition
  • Deep learning
  • Neural Networks
  • Similar gestures

Cite this

Wu, DI., Chen, J., Sharma, N., Pan, S., Long, G., & Blumenstein, M. (2019). Adversarial action data augmentation for similar gesture action recognition. In P. Angelov, & M. Roveri (Eds.), International Joint Conference on Neural Networks (IJCNN) 2019 [8851993] Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/IJCNN.2019.8851993
Wu, DI ; Chen, Junjun ; Sharma, Nabin ; Pan, Shirui ; Long, Guodong ; Blumenstein, Michael. / Adversarial action data augmentation for similar gesture action recognition. International Joint Conference on Neural Networks (IJCNN) 2019. editor / Plamen Angelov ; Manuel Roveri. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2019.
@inproceedings{b0e21a7c55174208b1be6d82e8dbc18a,
title = "Adversarial action data augmentation for similar gesture action recognition",
abstract = "Human gestures are unique for recognizing and describing human actions, and video-based human action recognition techniques are effective solutions to varies real-world applications, such as surveillance, video indexing, and human-computer interaction. Most existing video human action recognition approaches either using handcraft features from the frames or deep learning models such as convolutional neural networks (CNN) and recurrent neural networks (RNN); however, they have mostly overlooked the similar gestures between different actions when processing the frames into the models. The classifiers suffer from similar features extracted from similar gestures, which are unable to classify the actions in the video streams. In this paper, we propose a novel framework with generative adversarial networks (GAN) to generate the data augmentation for similar gesture action recognition. The contribution of our work is tri-fold: 1) we proposed a novel action data augmentation framework (ADAF) to enlarge the differences between the actions with very similar gestures; 2) the framework can boost the classification performance either on similar gesture action pairs or the whole dataset; 3) experiments conducted on both KTH and UCF101 datasets show that our data augmentation framework boost the performance on both similar gestures actions as well as the whole dataset compared with baseline methods such as 2DCNN and 3DCNN.",
keywords = "Action recognition, Deep learning, Neural Networks, Similar gestures",
author = "DI Wu and Junjun Chen and Nabin Sharma and Shirui Pan and Guodong Long and Michael Blumenstein",
year = "2019",
doi = "10.1109/IJCNN.2019.8851993",
language = "English",
isbn = "9781728119861",
editor = "Plamen Angelov and Manuel Roveri",
booktitle = "International Joint Conference on Neural Networks (IJCNN) 2019",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
address = "United States of America",

}

Wu, DI, Chen, J, Sharma, N, Pan, S, Long, G & Blumenstein, M 2019, Adversarial action data augmentation for similar gesture action recognition. in P Angelov & M Roveri (eds), International Joint Conference on Neural Networks (IJCNN) 2019., 8851993, IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, IEEE International Joint Conference on Neural Networks 2019, Budapest, Hungary, 14/07/19. https://doi.org/10.1109/IJCNN.2019.8851993

Adversarial action data augmentation for similar gesture action recognition. / Wu, DI; Chen, Junjun; Sharma, Nabin; Pan, Shirui; Long, Guodong; Blumenstein, Michael.

International Joint Conference on Neural Networks (IJCNN) 2019. ed. / Plamen Angelov; Manuel Roveri. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2019. 8851993.

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - Adversarial action data augmentation for similar gesture action recognition

AU - Wu, DI

AU - Chen, Junjun

AU - Sharma, Nabin

AU - Pan, Shirui

AU - Long, Guodong

AU - Blumenstein, Michael

PY - 2019

Y1 - 2019

N2 - Human gestures are unique for recognizing and describing human actions, and video-based human action recognition techniques are effective solutions to varies real-world applications, such as surveillance, video indexing, and human-computer interaction. Most existing video human action recognition approaches either using handcraft features from the frames or deep learning models such as convolutional neural networks (CNN) and recurrent neural networks (RNN); however, they have mostly overlooked the similar gestures between different actions when processing the frames into the models. The classifiers suffer from similar features extracted from similar gestures, which are unable to classify the actions in the video streams. In this paper, we propose a novel framework with generative adversarial networks (GAN) to generate the data augmentation for similar gesture action recognition. The contribution of our work is tri-fold: 1) we proposed a novel action data augmentation framework (ADAF) to enlarge the differences between the actions with very similar gestures; 2) the framework can boost the classification performance either on similar gesture action pairs or the whole dataset; 3) experiments conducted on both KTH and UCF101 datasets show that our data augmentation framework boost the performance on both similar gestures actions as well as the whole dataset compared with baseline methods such as 2DCNN and 3DCNN.

AB - Human gestures are unique for recognizing and describing human actions, and video-based human action recognition techniques are effective solutions to varies real-world applications, such as surveillance, video indexing, and human-computer interaction. Most existing video human action recognition approaches either using handcraft features from the frames or deep learning models such as convolutional neural networks (CNN) and recurrent neural networks (RNN); however, they have mostly overlooked the similar gestures between different actions when processing the frames into the models. The classifiers suffer from similar features extracted from similar gestures, which are unable to classify the actions in the video streams. In this paper, we propose a novel framework with generative adversarial networks (GAN) to generate the data augmentation for similar gesture action recognition. The contribution of our work is tri-fold: 1) we proposed a novel action data augmentation framework (ADAF) to enlarge the differences between the actions with very similar gestures; 2) the framework can boost the classification performance either on similar gesture action pairs or the whole dataset; 3) experiments conducted on both KTH and UCF101 datasets show that our data augmentation framework boost the performance on both similar gestures actions as well as the whole dataset compared with baseline methods such as 2DCNN and 3DCNN.

KW - Action recognition

KW - Deep learning

KW - Neural Networks

KW - Similar gestures

UR - http://www.scopus.com/inward/record.url?scp=85073259331&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2019.8851993

DO - 10.1109/IJCNN.2019.8851993

M3 - Conference Paper

SN - 9781728119861

BT - International Joint Conference on Neural Networks (IJCNN) 2019

A2 - Angelov, Plamen

A2 - Roveri, Manuel

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

ER -

Wu DI, Chen J, Sharma N, Pan S, Long G, Blumenstein M. Adversarial action data augmentation for similar gesture action recognition. In Angelov P, Roveri M, editors, International Joint Conference on Neural Networks (IJCNN) 2019. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2019. 8851993 https://doi.org/10.1109/IJCNN.2019.8851993