Automatic eye gaze estimation using geometric texture-based networks

Shreyank Jyoti, Abhinav Dhall

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

1 Citation (Scopus)

Abstract

Eye gaze estimation is an important problem in automatic human behavior understanding. This paper proposes a deep learning based method for inferring the eye gaze direction. The method is based on the use of ensemble of networks, which capture both the geometric and texture information. Firstly, a Deep Neural Network (DNN) is trained using the geometric features that are extracted from the facial landmark locations. Secondly, for the texture based features, three Convolutional Neural Networks (CNN) are trained i.e. For the patch around the left eye, right eye, and the combined eyes, respectively. Finally, the information from the four channels is fused with concatenation and dense layers are trained to predict the final eye gaze. The experiments are performed on the two publicly available datasets: Columbia eye gaze and TabletGaze. The extensive evaluation shows the superior performance of the proposed framework. We also evaluate the performance of the recently proposed swish activation function as compared to Rectified Linear Unit (ReLU) for eye gaze estimation.

Original languageEnglish
Title of host publication2018 24th International Conference on Pattern Recognition (ICPR)
Subtitle of host publicationAug. 20 2018 to Aug. 24 2018 Beijing, China
EditorsCheng-Lin Liu, Rama Chellappa, Matti Pietikäinen
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages2474-2479
Number of pages6
ISBN (Electronic)9781538637883, 9781538637876
ISBN (Print)9781538637890
DOIs
Publication statusPublished - 2018
Externally publishedYes
EventInternational Conference on Pattern Recognition 2018 - Beijing, China
Duration: 20 Aug 201824 Aug 2018
Conference number: 24th
http://www.icpr2018.org/

Conference

ConferenceInternational Conference on Pattern Recognition 2018
Abbreviated titleICPR 2018
CountryChina
CityBeijing
Period20/08/1824/08/18
Internet address

Cite this

Jyoti, S., & Dhall, A. (2018). Automatic eye gaze estimation using geometric texture-based networks. In C-L. Liu, R. Chellappa, & M. Pietikäinen (Eds.), 2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China (pp. 2474-2479). [8545162] Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICPR.2018.8545162
Jyoti, Shreyank ; Dhall, Abhinav. / Automatic eye gaze estimation using geometric texture-based networks. 2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China. editor / Cheng-Lin Liu ; Rama Chellappa ; Matti Pietikäinen. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. pp. 2474-2479
@inproceedings{3bcda7967b684ee9bc683a33975c897c,
title = "Automatic eye gaze estimation using geometric texture-based networks",
abstract = "Eye gaze estimation is an important problem in automatic human behavior understanding. This paper proposes a deep learning based method for inferring the eye gaze direction. The method is based on the use of ensemble of networks, which capture both the geometric and texture information. Firstly, a Deep Neural Network (DNN) is trained using the geometric features that are extracted from the facial landmark locations. Secondly, for the texture based features, three Convolutional Neural Networks (CNN) are trained i.e. For the patch around the left eye, right eye, and the combined eyes, respectively. Finally, the information from the four channels is fused with concatenation and dense layers are trained to predict the final eye gaze. The experiments are performed on the two publicly available datasets: Columbia eye gaze and TabletGaze. The extensive evaluation shows the superior performance of the proposed framework. We also evaluate the performance of the recently proposed swish activation function as compared to Rectified Linear Unit (ReLU) for eye gaze estimation.",
author = "Shreyank Jyoti and Abhinav Dhall",
year = "2018",
doi = "10.1109/ICPR.2018.8545162",
language = "English",
isbn = "9781538637890",
pages = "2474--2479",
editor = "Cheng-Lin Liu and Chellappa, {Rama } and Pietik{\"a}inen, {Matti }",
booktitle = "2018 24th International Conference on Pattern Recognition (ICPR)",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
address = "United States of America",

}

Jyoti, S & Dhall, A 2018, Automatic eye gaze estimation using geometric texture-based networks. in C-L Liu, R Chellappa & M Pietikäinen (eds), 2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China., 8545162, IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 2474-2479, International Conference on Pattern Recognition 2018, Beijing, China, 20/08/18. https://doi.org/10.1109/ICPR.2018.8545162

Automatic eye gaze estimation using geometric texture-based networks. / Jyoti, Shreyank; Dhall, Abhinav.

2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China. ed. / Cheng-Lin Liu; Rama Chellappa; Matti Pietikäinen. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. p. 2474-2479 8545162.

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - Automatic eye gaze estimation using geometric texture-based networks

AU - Jyoti, Shreyank

AU - Dhall, Abhinav

PY - 2018

Y1 - 2018

N2 - Eye gaze estimation is an important problem in automatic human behavior understanding. This paper proposes a deep learning based method for inferring the eye gaze direction. The method is based on the use of ensemble of networks, which capture both the geometric and texture information. Firstly, a Deep Neural Network (DNN) is trained using the geometric features that are extracted from the facial landmark locations. Secondly, for the texture based features, three Convolutional Neural Networks (CNN) are trained i.e. For the patch around the left eye, right eye, and the combined eyes, respectively. Finally, the information from the four channels is fused with concatenation and dense layers are trained to predict the final eye gaze. The experiments are performed on the two publicly available datasets: Columbia eye gaze and TabletGaze. The extensive evaluation shows the superior performance of the proposed framework. We also evaluate the performance of the recently proposed swish activation function as compared to Rectified Linear Unit (ReLU) for eye gaze estimation.

AB - Eye gaze estimation is an important problem in automatic human behavior understanding. This paper proposes a deep learning based method for inferring the eye gaze direction. The method is based on the use of ensemble of networks, which capture both the geometric and texture information. Firstly, a Deep Neural Network (DNN) is trained using the geometric features that are extracted from the facial landmark locations. Secondly, for the texture based features, three Convolutional Neural Networks (CNN) are trained i.e. For the patch around the left eye, right eye, and the combined eyes, respectively. Finally, the information from the four channels is fused with concatenation and dense layers are trained to predict the final eye gaze. The experiments are performed on the two publicly available datasets: Columbia eye gaze and TabletGaze. The extensive evaluation shows the superior performance of the proposed framework. We also evaluate the performance of the recently proposed swish activation function as compared to Rectified Linear Unit (ReLU) for eye gaze estimation.

UR - http://www.scopus.com/inward/record.url?scp=85059762588&partnerID=8YFLogxK

U2 - 10.1109/ICPR.2018.8545162

DO - 10.1109/ICPR.2018.8545162

M3 - Conference Paper

SN - 9781538637890

SP - 2474

EP - 2479

BT - 2018 24th International Conference on Pattern Recognition (ICPR)

A2 - Liu, Cheng-Lin

A2 - Chellappa, Rama

A2 - Pietikäinen, Matti

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

ER -

Jyoti S, Dhall A. Automatic eye gaze estimation using geometric texture-based networks. In Liu C-L, Chellappa R, Pietikäinen M, editors, 2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2018. p. 2474-2479. 8545162 https://doi.org/10.1109/ICPR.2018.8545162