From prediction to impact: evaluation of a learning analytics retention program

Shane Dawson, Jelena Jovanovic, Dragan Gašević, Abelardo Pardo

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

16 Citations (Scopus)

Abstract

Learning analytics research has often been touted as a means to address concerns regarding student retention outcomes. However, few research studies to date, have examined the impact of the implemented intervention strategies designed to address such retention challenges. Moreover, the methodological rigor of some of the existing studies has been challenged. This study evaluates the impact of a pilot retention program. The study contrasts the findings obtained by the use of different methods for analysis of the effect of the intervention. The pilot study was undertaken between 2012 and 2014 resulting in a combined enrolment of 11,160 students. A model to predict attrition was developed, drawing on data from student information system, learning management system interactions, and assessment. The predictive model identified some 1868 students as academically at-risk. Early interventions were implemented involving learning and remediation support. Common statistical methods demonstrated a positive association between the intervention and student retention. However, the effect size was low. The use of more advanced statistical methods, specifically mixed-effect methods explained higher variability in the data (over 99%), yet found the intervention had no effect on the retention outcomes. The study demonstrates that more data about individual differences is required to not only explain retention but to also develop more effective intervention approaches.

Original languageEnglish
Title of host publicationLAK'17 Conference Proceedings
Subtitle of host publicationThe Seventh International Learning Analytics and Knowledge Conference: Understanding, Informing and Improving Learning with Data
EditorsInge Molenaar, Xavier Ochoa, Shane Dawson
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages474-478
Number of pages5
ISBN (Electronic)9781450348706
DOIs
Publication statusPublished - 13 Mar 2017
Externally publishedYes
EventInternational Learning Analytics & Knowledge Conference 2017 - Morris J Wosk Centre for Dialogue, Simon Fraser University, Vancouver, Canada
Duration: 13 Mar 201717 Mar 2017
Conference number: 7th
http://lak17.solaresearch.org/

Conference

ConferenceInternational Learning Analytics & Knowledge Conference 2017
Abbreviated titleLAK 2017
CountryCanada
CityVancouver
Period13/03/1717/03/17
Internet address

Keywords

  • Early alert systems
  • Learning analytics
  • Mixed-effects model
  • Predictive models
  • Student retention

Cite this

Dawson, S., Jovanovic, J., Gašević, D., & Pardo, A. (2017). From prediction to impact: evaluation of a learning analytics retention program. In I. Molenaar, X. Ochoa, & S. Dawson (Eds.), LAK'17 Conference Proceedings: The Seventh International Learning Analytics and Knowledge Conference: Understanding, Informing and Improving Learning with Data (pp. 474-478). Association for Computing Machinery (ACM). https://doi.org/10.1145/3027385.3027405