Abstract
Human gestures are unique for recognizing and describing human actions, and video-based human action recognition techniques are effective solutions to varies real-world applications, such as surveillance, video indexing, and human-computer interaction. Most existing video human action recognition approaches either using handcraft features from the frames or deep learning models such as convolutional neural networks (CNN) and recurrent neural networks (RNN); however, they have mostly overlooked the similar gestures between different actions when processing the frames into the models. The classifiers suffer from similar features extracted from similar gestures, which are unable to classify the actions in the video streams. In this paper, we propose a novel framework with generative adversarial networks (GAN) to generate the data augmentation for similar gesture action recognition. The contribution of our work is tri-fold: 1) we proposed a novel action data augmentation framework (ADAF) to enlarge the differences between the actions with very similar gestures; 2) the framework can boost the classification performance either on similar gesture action pairs or the whole dataset; 3) experiments conducted on both KTH and UCF101 datasets show that our data augmentation framework boost the performance on both similar gestures actions as well as the whole dataset compared with baseline methods such as 2DCNN and 3DCNN.
| Original language | English |
|---|---|
| Title of host publication | International Joint Conference on Neural Networks (IJCNN) 2019 |
| Editors | Plamen Angelov, Manuel Roveri |
| Place of Publication | Piscataway NJ USA |
| Publisher | IEEE, Institute of Electrical and Electronics Engineers |
| Number of pages | 8 |
| ISBN (Electronic) | 9781728119854 |
| ISBN (Print) | 9781728119861 |
| DOIs | |
| Publication status | Published - 2019 |
| Event | IEEE International Joint Conference on Neural Networks 2019 - Budapest, Hungary Duration: 14 Jul 2019 → 19 Jul 2019 https://www.ijcnn.org/ |
Conference
| Conference | IEEE International Joint Conference on Neural Networks 2019 |
|---|---|
| Abbreviated title | IJCNN 2019 |
| Country | Hungary |
| City | Budapest |
| Period | 14/07/19 → 19/07/19 |
| Internet address |
Keywords
- Action recognition
- Deep learning
- Neural Networks
- Similar gestures
Cite this
}
Adversarial action data augmentation for similar gesture action recognition. / Wu, DI; Chen, Junjun; Sharma, Nabin; Pan, Shirui; Long, Guodong; Blumenstein, Michael.
International Joint Conference on Neural Networks (IJCNN) 2019. ed. / Plamen Angelov; Manuel Roveri. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2019. 8851993.Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review
TY - GEN
T1 - Adversarial action data augmentation for similar gesture action recognition
AU - Wu, DI
AU - Chen, Junjun
AU - Sharma, Nabin
AU - Pan, Shirui
AU - Long, Guodong
AU - Blumenstein, Michael
PY - 2019
Y1 - 2019
N2 - Human gestures are unique for recognizing and describing human actions, and video-based human action recognition techniques are effective solutions to varies real-world applications, such as surveillance, video indexing, and human-computer interaction. Most existing video human action recognition approaches either using handcraft features from the frames or deep learning models such as convolutional neural networks (CNN) and recurrent neural networks (RNN); however, they have mostly overlooked the similar gestures between different actions when processing the frames into the models. The classifiers suffer from similar features extracted from similar gestures, which are unable to classify the actions in the video streams. In this paper, we propose a novel framework with generative adversarial networks (GAN) to generate the data augmentation for similar gesture action recognition. The contribution of our work is tri-fold: 1) we proposed a novel action data augmentation framework (ADAF) to enlarge the differences between the actions with very similar gestures; 2) the framework can boost the classification performance either on similar gesture action pairs or the whole dataset; 3) experiments conducted on both KTH and UCF101 datasets show that our data augmentation framework boost the performance on both similar gestures actions as well as the whole dataset compared with baseline methods such as 2DCNN and 3DCNN.
AB - Human gestures are unique for recognizing and describing human actions, and video-based human action recognition techniques are effective solutions to varies real-world applications, such as surveillance, video indexing, and human-computer interaction. Most existing video human action recognition approaches either using handcraft features from the frames or deep learning models such as convolutional neural networks (CNN) and recurrent neural networks (RNN); however, they have mostly overlooked the similar gestures between different actions when processing the frames into the models. The classifiers suffer from similar features extracted from similar gestures, which are unable to classify the actions in the video streams. In this paper, we propose a novel framework with generative adversarial networks (GAN) to generate the data augmentation for similar gesture action recognition. The contribution of our work is tri-fold: 1) we proposed a novel action data augmentation framework (ADAF) to enlarge the differences between the actions with very similar gestures; 2) the framework can boost the classification performance either on similar gesture action pairs or the whole dataset; 3) experiments conducted on both KTH and UCF101 datasets show that our data augmentation framework boost the performance on both similar gestures actions as well as the whole dataset compared with baseline methods such as 2DCNN and 3DCNN.
KW - Action recognition
KW - Deep learning
KW - Neural Networks
KW - Similar gestures
UR - http://www.scopus.com/inward/record.url?scp=85073259331&partnerID=8YFLogxK
U2 - 10.1109/IJCNN.2019.8851993
DO - 10.1109/IJCNN.2019.8851993
M3 - Conference Paper
SN - 9781728119861
BT - International Joint Conference on Neural Networks (IJCNN) 2019
A2 - Angelov, Plamen
A2 - Roveri, Manuel
PB - IEEE, Institute of Electrical and Electronics Engineers
CY - Piscataway NJ USA
ER -