Assessing algorithmic fairness in automatic classifiers of educational forum posts

Lele Sha, Mladen Rakovic, Alex Whitelock-Wainwright, David Carroll, Victoria Yew, Dragan Gašević, Guanliang Chen

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

7 Citations (Scopus)

Abstract

Automatic classifiers of educational forum posts are essential in helping instructors effectively implement their teaching practices and thus have been widely investigated. However, existing studies mostly stressed the accuracy of a classifier, while the fairness of the classifier remains largely unexplored, i.e., whether the posts generated by a group of students are more likely to be correctly labeled than those generated by other groups of students. Undoubtedly, any unfairness based on student performance, sex, or other subjective views can have a detrimental effect on a student’s learning experience and performance. Therefore, this study aimed to assess the algorithmic fairness of six popular models used in building automatic classifiers of educational forum posts. Here, we measured the algorithmic fairness displayed (i) between students of different sex (female vs. male) and (ii) between students of different first languages (English-as-first-language speakers vs. English-as-second-language speakers). Besides, we investigated whether a classifier’s fairness could be enhanced by applying data sampling techniques. Our results demonstrated that: 1) traditional Machine Learning models slightly outperformed up-to-date Deep Learning models in delivering fair predictions; 2) students of different first languages faced more unfair predictions than students of different sex, and most of the classifiers tended to favor English-as-first-language students; and 3) with equal numbers of posts generated by different groups of students in the training data, the fairness of a classifier could be greatly enhanced.

Original languageEnglish
Title of host publicationArtificial Intelligence in Education
Subtitle of host publication22nd International Conference, AIED 2021 Utrecht, The Netherlands, June 14–18, 2021 Proceedings, Part I
EditorsIdo Roll, Danielle McNamara, Sergey Sosnovsky, Rose Luckin, Vania Dimitrova
Place of PublicationCham Switzerland
PublisherSpringer
Pages381-394
Number of pages14
ISBN (Electronic)9783030782924
ISBN (Print)9783030782917
DOIs
Publication statusPublished - 2021
EventInternational Conference on Artificial Intelligence in Education 2021 - Utrecht, Netherlands
Duration: 14 Jun 202118 Jun 2021
Conference number: 22nd
https://link.springer.com/book/10.1007/978-3-030-78292-4 (Proceedings)

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume12748
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Conference on Artificial Intelligence in Education 2021
Abbreviated titleAIED 2021
Country/TerritoryNetherlands
CityUtrecht
Period14/06/2118/06/21
Internet address

Keywords

  • Algorithmic fairness
  • Educational forum post
  • Text classification

Cite this