Prediction and localization of student engagement in the wild

Amanjot Kaur, Aamir Mustafa, Love Mehta, Abhinav Dhall

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

21 Citations (Scopus)


Digital revolution has transformed the traditional teaching procedures, students are going online to access study materials. It is realised that analysis of student engagement in an e-learning environment would facilitate effective task accomplishment and learning. Well known social cues of engagement/disengagement can be inferred from facial expressions, body movements and gaze patterns. In this paper, student's response to various stimuli (educational videos) are recorded and cues are extracted to estimate variations in engagement level. We study the association of a subject's behavioral cues with his/her engagement level, as annotated by labelers. We have localized engaging/non-engaging parts in the stimuli videos using a deep multiple instance learning based framework, which can give useful insight into designing Massive Open Online Courses (MOOCs) video material. Recognizing the lack of any publicly available dataset in the domain of user engagement, a new 'in the wild' dataset is curated. The dataset: Engagement in the Wild contains 264 videos captured from 91 subjects, which is approximately 16.5 hours of recording. Detailed baseline results using different classifiers ranging from traditional machine learning to deep learning based approaches are evaluated on the database. Subject independent analysis is performed and the task of engagement prediction is modeled as a weakly supervised learning problem. The dataset is manually annotated by different labelers and the correlation studies between annotated and predicted labels of videos by different classifiers are reported. This dataset creation is an effort to facilitate research in various e-learning environments such as intelligent tutoring systems, MOOCs, and others.

Original languageEnglish
Title of host publication2018 International Conference on Digital Image Computing: Techniques and Applications (DICTA)
EditorsManzur Murshed, Manoranjan Paul, Md Asikuzzaman, Mark Pickering, Ambarish Natu, Antonio Robles-Kelly, Shaodi You, Lihong Zheng, Ashfaqur Rahman
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Electronic)9781538666029, 9781538666012
ISBN (Print)9781538666036
Publication statusPublished - 2018
Externally publishedYes
EventDigital Image Computing Techniques and Applications 2018 - Canberra, Australia
Duration: 10 Dec 201813 Dec 2018
Conference number: 20th (Proceedings)


ConferenceDigital Image Computing Techniques and Applications 2018
Abbreviated titleDICTA 2018
Internet address


  • Dataset for Student Engagement
  • E-learning Environment
  • Engagement Detection
  • Engagement Localization

Cite this