What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course

Lisa Angelique Lim, Sheridan Gentili, Abelardo Pardo, Vitomir Kovanović, Alexander Whitelock-Wainwright, Dragan Gašević, Shane Dawson

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Recent developments in educational technologies have provided a viable solution to the challenges associated with scaling personalised feedback to students. However, there is currently little empirical evidence about the impact such scaled feedback has on student learning progress and study behaviour. This paper presents the findings of a study that looked at the impact of a learning analytics (LA)-based feedback system on students' self-regulated learning and academic achievement in a large, first-year undergraduate course. Using the COPES model of self-regulated learning (SRL), we analysed the learning operations of students, by way of log data from the learning management system and e-book, as well as the products of SRL, namely, performance on course assessments, from three years of course offerings. The latest course offering involved an intervention condition that made use of an educational technology to provide LA-based process feedback. Propensity score matching was employed to match a control group to the student cohort enrolled in the latest course offering, creating two equal-sized groups of students who received the feedback (the experimental group) and those who did not (the control group). Growth mixture modelling and mixed between-within ANOVA were also employed to identify differences in the patterns of online self-regulated learning operations over the course of the semester. The results showed that the experimental group showed significantly different patterns in their learning operations and performed better in terms of final grades. Moreover, there was no difference in the effect of feedback on final grades among students with different prior academic achievement scores, indicating that the LA-based feedback deployed in this course is able to support students’ learning, regardless of prior academic standing.

Original languageEnglish
Number of pages11
JournalLearning and Instruction
DOIs
Publication statusAccepted/In press - May 2019

Keywords

  • Feedback
  • Higher education
  • Large enrolment courses
  • Learning analytics
  • Self-regulated learning

Cite this

@article{88903a010abc4df4a6dfce392c64417d,
title = "What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course",
abstract = "Recent developments in educational technologies have provided a viable solution to the challenges associated with scaling personalised feedback to students. However, there is currently little empirical evidence about the impact such scaled feedback has on student learning progress and study behaviour. This paper presents the findings of a study that looked at the impact of a learning analytics (LA)-based feedback system on students' self-regulated learning and academic achievement in a large, first-year undergraduate course. Using the COPES model of self-regulated learning (SRL), we analysed the learning operations of students, by way of log data from the learning management system and e-book, as well as the products of SRL, namely, performance on course assessments, from three years of course offerings. The latest course offering involved an intervention condition that made use of an educational technology to provide LA-based process feedback. Propensity score matching was employed to match a control group to the student cohort enrolled in the latest course offering, creating two equal-sized groups of students who received the feedback (the experimental group) and those who did not (the control group). Growth mixture modelling and mixed between-within ANOVA were also employed to identify differences in the patterns of online self-regulated learning operations over the course of the semester. The results showed that the experimental group showed significantly different patterns in their learning operations and performed better in terms of final grades. Moreover, there was no difference in the effect of feedback on final grades among students with different prior academic achievement scores, indicating that the LA-based feedback deployed in this course is able to support students’ learning, regardless of prior academic standing.",
keywords = "Feedback, Higher education, Large enrolment courses, Learning analytics, Self-regulated learning",
author = "Lim, {Lisa Angelique} and Sheridan Gentili and Abelardo Pardo and Vitomir Kovanović and Alexander Whitelock-Wainwright and Dragan Gašević and Shane Dawson",
year = "2019",
month = "5",
doi = "10.1016/j.learninstruc.2019.04.003",
language = "English",
journal = "Learning and Instruction",
issn = "0959-4752",
publisher = "Elsevier",

}

What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course. / Lim, Lisa Angelique; Gentili, Sheridan; Pardo, Abelardo; Kovanović, Vitomir; Whitelock-Wainwright, Alexander; Gašević, Dragan; Dawson, Shane.

In: Learning and Instruction, 05.2019.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course

AU - Lim, Lisa Angelique

AU - Gentili, Sheridan

AU - Pardo, Abelardo

AU - Kovanović, Vitomir

AU - Whitelock-Wainwright, Alexander

AU - Gašević, Dragan

AU - Dawson, Shane

PY - 2019/5

Y1 - 2019/5

N2 - Recent developments in educational technologies have provided a viable solution to the challenges associated with scaling personalised feedback to students. However, there is currently little empirical evidence about the impact such scaled feedback has on student learning progress and study behaviour. This paper presents the findings of a study that looked at the impact of a learning analytics (LA)-based feedback system on students' self-regulated learning and academic achievement in a large, first-year undergraduate course. Using the COPES model of self-regulated learning (SRL), we analysed the learning operations of students, by way of log data from the learning management system and e-book, as well as the products of SRL, namely, performance on course assessments, from three years of course offerings. The latest course offering involved an intervention condition that made use of an educational technology to provide LA-based process feedback. Propensity score matching was employed to match a control group to the student cohort enrolled in the latest course offering, creating two equal-sized groups of students who received the feedback (the experimental group) and those who did not (the control group). Growth mixture modelling and mixed between-within ANOVA were also employed to identify differences in the patterns of online self-regulated learning operations over the course of the semester. The results showed that the experimental group showed significantly different patterns in their learning operations and performed better in terms of final grades. Moreover, there was no difference in the effect of feedback on final grades among students with different prior academic achievement scores, indicating that the LA-based feedback deployed in this course is able to support students’ learning, regardless of prior academic standing.

AB - Recent developments in educational technologies have provided a viable solution to the challenges associated with scaling personalised feedback to students. However, there is currently little empirical evidence about the impact such scaled feedback has on student learning progress and study behaviour. This paper presents the findings of a study that looked at the impact of a learning analytics (LA)-based feedback system on students' self-regulated learning and academic achievement in a large, first-year undergraduate course. Using the COPES model of self-regulated learning (SRL), we analysed the learning operations of students, by way of log data from the learning management system and e-book, as well as the products of SRL, namely, performance on course assessments, from three years of course offerings. The latest course offering involved an intervention condition that made use of an educational technology to provide LA-based process feedback. Propensity score matching was employed to match a control group to the student cohort enrolled in the latest course offering, creating two equal-sized groups of students who received the feedback (the experimental group) and those who did not (the control group). Growth mixture modelling and mixed between-within ANOVA were also employed to identify differences in the patterns of online self-regulated learning operations over the course of the semester. The results showed that the experimental group showed significantly different patterns in their learning operations and performed better in terms of final grades. Moreover, there was no difference in the effect of feedback on final grades among students with different prior academic achievement scores, indicating that the LA-based feedback deployed in this course is able to support students’ learning, regardless of prior academic standing.

KW - Feedback

KW - Higher education

KW - Large enrolment courses

KW - Learning analytics

KW - Self-regulated learning

UR - http://www.scopus.com/inward/record.url?scp=85065618970&partnerID=8YFLogxK

U2 - 10.1016/j.learninstruc.2019.04.003

DO - 10.1016/j.learninstruc.2019.04.003

M3 - Article

JO - Learning and Instruction

JF - Learning and Instruction

SN - 0959-4752

ER -