Towards the Automated Generation of Readily Applicable Personalised Feedback in Education

Providing personalised feedback to a large student cohort is a longstanding challenge in education. Recent work in prescriptive learning analytics (PLA) demonstrated a promising approach by augmenting predictive models with prescriptive capabilities of explainable artificial intelligence (XAI). Although theoretically sound, in practice, not all predictive features can be leveraged by XAI to prescribe useful feedback. It remains under-explored as to how to engineer such predictive features that can be used to prescribe personalised and actionable feedback. To address this, we proposed a learning activity-based approach to design features that are informative to both predictive and prescriptive performance in PLA. We conducted empirical evaluations of the quality of PLA-generated feedback compared to feedback written by experienced teachers in a large-scale university course. Four rubric criteria, including Readily Applicablility, Readability, Relational, and Specificity, were designed based on previous research. We found that: (i) By adopting learning activity-based features, PLA generates high quality feedback without sacrificing predictive performance; (ii) Most experienced teaching staff rated PLA-generated feedback as readily applicable to the course; and (iii) Compared to teacher-written feedback, the quality of PLA-generated feedback is consistently rated higher (with statistical significance) in all four rubric criteria by experienced teachers. All code is available via our GitHub repository (https://github.com/CoLAMZP/AIED-2024-AutoFeedback).