Hybrid neural networks based approach for Holoscopic Micro-Gesture Recognition in images and videos

Garima Sharma; Shreyank Jyoti; Abhinav Dhall

doi:10.1109/FG.2018.00130

Hybrid neural networks based approach for Holoscopic Micro-Gesture Recognition in images and videos

Garima Sharma, Shreyank Jyoti, Abhinav Dhall

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

1 Citation (Scopus)

Abstract

This paper presents an approach for hand based micro-gesture recognition in images and videos as part of the Holoscopic Micro-Gesture Recognition (HoMGR) challenge. The database consists of Holoscopic 3D Micro-Gesture images and videos. The proposed framework is an ensemble of convolutional neural network and deep neural network. The framework performs feature fusion technique on both handcrafted (local phase quantization) and deep features extracted from the neural network, to leverage on complimentary information. The powerful discriminative nature of the fused features has proved beneficial on the given HoMGR challenge data. The experiments show that the proposed approach is effective and outperforms the baseline on the Test set by an absolute margin of 26.67% for images and 2.47% for videos, respectively.

Original language	English
Title of host publication	Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018
Subtitle of host publication	15–19 May 2018 Xi’an, China
Editors	Sidney D’Mello, Louis‐Philippe Morency, Michel Valstar, Lijun Yin
Place of Publication	Piscataway NJ USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	808-814
Number of pages	7
ISBN (Electronic)	9781538623350
ISBN (Print)	9781538623367
DOIs	https://doi.org/10.1109/FG.2018.00130
Publication status	Published - 2018
Externally published	Yes
Event	IEEE International Conference on Automatic Face and Gesture Recognition 2018 - Xi'an, China Duration: 15 May 2018 → 19 May 2018 Conference number: 13th https://fg2018.cse.sc.edu/

Conference

Conference	IEEE International Conference on Automatic Face and Gesture Recognition 2018
Abbreviated title	FG 2018
Country	China
City	Xi'an
Period	15/05/18 → 19/05/18
Internet address	https://fg2018.cse.sc.edu/

Keywords

Holoscopy
Micro gesture recognition

Access to Document

10.1109/FG.2018.00130

Link to publication in Scopus

Cite this

Sharma, G., Jyoti, S., & Dhall, A. (2018). Hybrid neural networks based approach for Holoscopic Micro-Gesture Recognition in images and videos. In S. D’Mello, LP. Morency, M. Valstar, & L. Yin (Eds.), Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018: 15–19 May 2018 Xi’an, China (pp. 808-814). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/FG.2018.00130

Sharma, Garima ; Jyoti, Shreyank ; Dhall, Abhinav. / Hybrid neural networks based approach for Holoscopic Micro-Gesture Recognition in images and videos. Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018: 15–19 May 2018 Xi’an, China. editor / Sidney D’Mello ; Louis‐Philippe Morency ; Michel Valstar ; Lijun Yin. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. pp. 808-814

@inproceedings{8eeae51b7b704b5c885623b0752cde09,

title = "Hybrid neural networks based approach for Holoscopic Micro-Gesture Recognition in images and videos",

abstract = "This paper presents an approach for hand based micro-gesture recognition in images and videos as part of the Holoscopic Micro-Gesture Recognition (HoMGR) challenge. The database consists of Holoscopic 3D Micro-Gesture images and videos. The proposed framework is an ensemble of convolutional neural network and deep neural network. The framework performs feature fusion technique on both handcrafted (local phase quantization) and deep features extracted from the neural network, to leverage on complimentary information. The powerful discriminative nature of the fused features has proved beneficial on the given HoMGR challenge data. The experiments show that the proposed approach is effective and outperforms the baseline on the Test set by an absolute margin of 26.67% for images and 2.47% for videos, respectively.",

keywords = "Holoscopy, Micro gesture recognition",

author = "Garima Sharma and Shreyank Jyoti and Abhinav Dhall",

year = "2018",

doi = "10.1109/FG.2018.00130",

language = "English",

isbn = "9781538623367",

pages = "808--814",

editor = "Sidney D{\textquoteright}Mello and Morency, {Louis‐Philippe } and Valstar, {Michel } and Lijun Yin",

booktitle = "Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

address = "United States of America",

note = "IEEE International Conference on Automatic Face and Gesture Recognition 2018, FG 2018 ; Conference date: 15-05-2018 Through 19-05-2018",

url = "https://fg2018.cse.sc.edu/",

}

Sharma, G, Jyoti, S & Dhall, A 2018, Hybrid neural networks based approach for Holoscopic Micro-Gesture Recognition in images and videos. in S D’Mello, LP Morency, M Valstar & L Yin (eds), Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018: 15–19 May 2018 Xi’an, China. IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 808-814, IEEE International Conference on Automatic Face and Gesture Recognition 2018, Xi'an, China, 15/05/18. https://doi.org/10.1109/FG.2018.00130

Hybrid neural networks based approach for Holoscopic Micro-Gesture Recognition in images and videos. / Sharma, Garima; Jyoti, Shreyank; Dhall, Abhinav.

Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018: 15–19 May 2018 Xi’an, China. ed. / Sidney D’Mello; Louis‐Philippe Morency; Michel Valstar; Lijun Yin. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. p. 808-814.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Hybrid neural networks based approach for Holoscopic Micro-Gesture Recognition in images and videos

AU - Sharma, Garima

AU - Jyoti, Shreyank

AU - Dhall, Abhinav

N1 - Conference code: 13th

PY - 2018

Y1 - 2018

N2 - This paper presents an approach for hand based micro-gesture recognition in images and videos as part of the Holoscopic Micro-Gesture Recognition (HoMGR) challenge. The database consists of Holoscopic 3D Micro-Gesture images and videos. The proposed framework is an ensemble of convolutional neural network and deep neural network. The framework performs feature fusion technique on both handcrafted (local phase quantization) and deep features extracted from the neural network, to leverage on complimentary information. The powerful discriminative nature of the fused features has proved beneficial on the given HoMGR challenge data. The experiments show that the proposed approach is effective and outperforms the baseline on the Test set by an absolute margin of 26.67% for images and 2.47% for videos, respectively.

AB - This paper presents an approach for hand based micro-gesture recognition in images and videos as part of the Holoscopic Micro-Gesture Recognition (HoMGR) challenge. The database consists of Holoscopic 3D Micro-Gesture images and videos. The proposed framework is an ensemble of convolutional neural network and deep neural network. The framework performs feature fusion technique on both handcrafted (local phase quantization) and deep features extracted from the neural network, to leverage on complimentary information. The powerful discriminative nature of the fused features has proved beneficial on the given HoMGR challenge data. The experiments show that the proposed approach is effective and outperforms the baseline on the Test set by an absolute margin of 26.67% for images and 2.47% for videos, respectively.

KW - Holoscopy

KW - Micro gesture recognition

UR - http://www.scopus.com/inward/record.url?scp=85049411281&partnerID=8YFLogxK

U2 - 10.1109/FG.2018.00130

DO - 10.1109/FG.2018.00130

M3 - Conference Paper

AN - SCOPUS:85049411281

SN - 9781538623367

SP - 808

EP - 814

BT - Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018

A2 - D’Mello, Sidney

A2 - Morency, Louis‐Philippe

A2 - Valstar, Michel

A2 - Yin, Lijun

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

T2 - IEEE International Conference on Automatic Face and Gesture Recognition 2018

Y2 - 15 May 2018 through 19 May 2018

ER -

Sharma G, Jyoti S, Dhall A. Hybrid neural networks based approach for Holoscopic Micro-Gesture Recognition in images and videos. In D’Mello S, Morency LP, Valstar M, Yin L, editors, Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018: 15–19 May 2018 Xi’an, China. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2018. p. 808-814 https://doi.org/10.1109/FG.2018.00130