Predictors of student satisfaction: a large-scale study of human-human online tutorial dialogues

Guanliang Chen, David Lang, Rafael Ferreira, Dragan Gašević

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

3 Citations (Scopus)


For the development of successful human-agent dialogue-based tutoring systems, it is essential to understand what makes a human-human tutorial dialogue successful. While there has been much research on dialogue-based intelligent tutoring systems, there have been comparatively fewer studies on analyzing large-scale datasets of human-human online tutoring dialogues. A critical indicator of success of a tutoring dialogue can be student satisfaction, which is the focus of the study reported in the paper. Specifically, we used a large-scale dataset, which consisted of over 15,000 tutorial dialogues generated by human tutors and students in a mobile app-based tutoring service. An extensive analysis of the dataset was performed to identify factors relevant to student satisfaction in online tutoring systems. The study also engineered a set of 325 features as input to a Gradient Tree Boosting model to predict tutoring success. Experimental results revealed that (i) in a tutorial dialogue, factors such as efforts spent by both tutors and students, utterance informativeness and tutor responsiveness were positively correlated with student satisfaction; and (ii) Gradient Tree Boosting model could effectively predict tutoring success, especially with utterances from the later period of a dialogue, but more research effort is needed to improve the prediction performance.

Original languageEnglish
Title of host publicationProceedings of the 12th International Conference on Educational Data Mining
EditorsCollin F. Lynch, Agathe Merceron, Michel Desmarais, Roger Nkambou
Place of PublicationMontreal Canada
PublisherPresses De I Universite Du Quebec
Number of pages10
ISBN (Electronic)9781733673600
Publication statusPublished - 2019
EventEducational Data Mining 2019 - Montreal , Canada
Duration: 2 Jul 20195 Jul 2019
Conference number: 12th


ConferenceEducational Data Mining 2019
Abbreviated titleEDM 2019
Internet address


  • Educational dialogue analysis
  • Gradient tree boosting
  • Intelligent tutoring systems
  • Student satisfaction

Cite this